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Foreword 


U  Col  Michael  A.  Sehiefer  began  (his  research  project  with  a  narrowly 
focused  topic:  to  illuminate  ways  the  Air  Force  Reserve  Officer  Training 
Corps  (AFROTC)  might  alter  its  scholarship  allocation  process  to  consider 
each  applicant’s  long-term  propensity  to  stay  in  the  military.  He  has 
thoroughly  and  quantitatively  documented  AFROTC  implementation  op¬ 
tions  to  include  impacts,  costs,  and  benefits. 

In  the  course  of  this  effort,  additional  AFROTC  analysis  needs  became 
apparent.  Colonel  Sehiefer  identifies  these  needs.  He  dlso  provides  es¬ 
timates  of  their  impacts  on  AFROTC  plannin^and  budgeting.  Finally,  he 
suggests  solutions. 


Director 

Airpower  Research  Institute 


De5nJNIS\TT)^EW.  Col.^AR 


vii 


About  the 
Author 


Lt  Col  Michael  A.  Scltiefer 


l.l  Col  Michael  A.  Schiefer  complclcd  1 1 1  i s  study  while  assiuned  lo  tie 
Airpowet  Research  Instilnlc  (Aid).  Air  (hnycisily  Center  lor  Aerosp a.  - 
I  )<k  I  line.  Rcscanh,  and  Kdiu  a  I  ion  (Al  h  A I  )RK)  at  Maxwell  An  For  re  I 1  im 
(AI  R),  Alahama.  In  1979  <  ’olonrl  S<  hiclci  ipadnatcd  from  the  i  Inilcd  Slat-  - 
Ail  Force  Academy  with  a  bat  helm  ol  science  decree  in  mat  hemal  ics  lie- 
lii  si  assignment  allei  gradual  ion  was  a l  Kirt  land  AIR,  New  Mexico,  ai  i  he 
An  Fort  e  Weapons  Imboratorv  u  lnae  he  modeled  nneleai  shock  w.iv 
elleets  on  reer  trv  vehn  le  lia|ei  tones  Retwrrn  197b  and  1979  al  tint's 
An  hinr  School  ol  Aerospai  e  Mrdii  ine.  he  pioyaiumed  a  imiii'-ni  .ill1, 
emit  rolled  milling  machine  lo  iiianul.it  line  a  vanety  ol  medical  icse.neh 
hardware.  Me  earned  a  masler’s  device  in  operations  research  al  I  In  *  An 
Foiee  Institute  ol  Technology  m  1979  and  worked  until  1989  at  lb  ad 
quarters  Air  Training  Command  (A  I  < ')  desiynmu  and  pi o< hn  iny  a  var  id  \  <  .1 
eon  t  ml  and  feedback  tools  lor  Air  F<  me  technical  t  raimnu  rnan.iiiei  ■  lie 
earned  a  doe  I  or  al  decree  at  t  he  I  hiiversil  v  ol  Texas  at  Ansi  m  in  198b  !  1  ill  it 
I  989,  he  was  chief  ol  the  Kn  listed  Analysis  Ih  am  I  i  I  lepnt  \  ( 'hid  o(  St .  1 1) 
lor  Personnel.  I  le.idqnar  ter  s  I  IN  A  I- .  the  Pentagon  In  1989  the  comma  inlet 
o|  AIC  selected  Inm  lo  he  a  rescan  h  lellovv  at  AR!  and  nun  imenlK  alt*  mi 
Air  War  College  (  oloncl  Schielci  is  piesenllv  assigned  lo  llrndquaitM  • 
AIC.  He  has  a  son.  Mat t hew 


IV 


Preface 


In  .June  1989  a  special  study  group  commissioned  by  the  Air  Training 
Command  (AFC)  commander.  Lt  Gen  Robert  C.  Oaks,  reported  it  had 
developed  analytical  techniques  to  predict  an  individual's  propensity  to  stay 
(PTS)  in  the  Air  Force.  The  ultimate  objective  of  the  study  was  to  begin 
bringing  people  into  the  Air  Force  who  would  be  more  likely  to  remain  for 
the  long  term,  thereby  easing  pilot  and  engineering  retention  problems.  In 
March  of  1989  Headquarters  ATC  directed  this  researcher  to  investigate 
how  FI'S  information  might  be  infused  into  Air  Force  Reserve  Officer 
Training  Corps  (AFROTC)  scholarship  allocation  decisions.  The  Oaks 
special  study  group  suggested  an  optimization  approach.  While  that 
approach  is  conceptually  appealing,  this  report  concludes  that  the  benefits 
of  such  an  implementation  approach  would  not  justify  the  costs.  This  study 
offers  objective  and  subjective  alternatives  that  complement  current 
AFROTC  selection  procedures. 

In  the  course  of  my  research.  I  concluded  that  AFROTC  has  other  pressing 
analysis  requirements.  AFROTC  needs  a  dynamic  model  to  estimate  policy 
impacts  on  production  and  outlays.  Such  a  model  requires  improved 
estimates  of  scholarship  student  persistence  (continuation)  rates.  AFROTC 
also  needs  to  refine  the  technique  it  uses  to  forecast  tuition  inflation  rates. 
Finally,  scholarship  managers  need  to  immediately  begin  capturing  data 
on  cadets  who  lose  then  scholarships. 

I  would  like  to  thank  Or  Karl  Magyar,  research  adviser,  and  Dr  Richard 
Bailey,  editor,  from  AUCADRE  for  their  valuable  suggestions  to  improve  this 
manuscript.  Lt  Col  Manfred  Koc/.ur,  also  from  AUCADRE,  kept  me  above 
water  administratively.  Majors  Rob  Gaston,  Mark  Lewis,  and  Fred  Fisher, 
and  ('apt  Mitchell  Norton  spent  a  great  deal  of  their  time  educating  me  on 
the  AFROTC  sc  holarship  allocation  system. 

Finally.  I  am  indebted  to  Brig  Gen  John  J.  Salvadore,  Brig  Gen  Jeffrey  T. 
Ellis,  and  Col  J.  R.  Pond  for  providing  resources  and  for  encouraging  their 
staffs  to  listen  to  what  I  had  to  say. 

yrifckodl  Q .  Sc 

MICHAEL  A.  SCHIEFER,  Lt  Col.  USAF 

Research  Fellow 

Airpower  Research  Institute 


Chapter  1 

Introduction 


The  Air  Force  has  three  primary  organizations  that  (rain  and  screen 
individuals  who  wish  to  become  commissioned  officers:  the  United  States 
Air  Force  Academy  (USAFA),  located  in  Colorado  Springs,  Colorado:  the 
Officer  Training  School  (OTS)  in  San  Antonio.  Texas:  and  the  Air  Force 
Reserve  Officer  Training  Corps  (AFROTC)  at  Maxwell  AFB,  Alabama,  with 
detachments  located  on  nearly  150  college  campuses.  AFROTC  and  Cl'S 
are  subordinate  organizations  of  the  Air  Training  Command  (ATC).  The  Air 
Force  does  not  accept  all  applicants  lor  officer  training.  Each  officer  training 
organization  subjectively  and  objectively  considers  a  variety  of  information 
to  select  applicants.  In  1989  the  commander  of  ATC  directed  AFROTC  and 
OTS  to  consider  in  their  selection  processes  a  recently  developed  predictor 
of  an  individual’s  propensity  to  stay  (PI'S)  in  the  Air  Force.  Subsequently. 
Headquarters  Air  Force  Recruiting  Service.  Officer  Accessions,  directed  this 
researcher  to  investigate  methods  AFROTC  might  use  to  implement  PI'S 
information  into  its  officer  candidate  selection  process. 

Background 

'Hie  Air  Force  has  experienced  chronic  pilot  and  occasional  engineer 
retention  problems.  Pilot  losses  create  readiness  and  economic  concerns 
because  of  the  years  and  millions  of  training  dollars  needed  to  replace  each 
pilot  who  leaves  the  Air  Force.  Airline  hiring  and  Air  Force  salaries  and 
management  practices  are  some  postcommissioning  factors  that  influence 
pilot  retention  rates.  If  precommissioning  factors  such  as  childhood  ex¬ 
periences  also  influence  pilot  retention,  then,  some  pilots  will  be  more 
predisposed  to  an  Air  Force  career  than  others.  In  1988  the  ATC  com¬ 
mander,  Lt  Gen  Robert  C.  Oaks,  charged  a  study  group  to  investigate 
whether  precommissioning  attributes  could  be  identified  for  individuals 
with  a  high  propensity  to  stay  in  the  Air  Force.  Since  ATC  selects,  trains, 
and  commissions  most  Air  Force  officers  through  AFROTC  and  OTS,  the 
command  might  increase  officer  retention  through  its  candidate  selection 
policies.  On  1  June  1989  the  Oaks  study  group  reported  in  its  “Keeper 
Study"  that  retention  probability  was  “distinctly  higher"  for  officers  with  any 
of  the  following  precommissioning  characteristics: 

•  attended  a  moderate-cost  college  (as  opposed  to  a  high -cost  college) 

•  had  military  parents 

•  moved  frequently  as  children 


1 


•  grew  up  in  (he  south-central  United  States 

•  joined  the  Air  Force  for  security  (instead  of  training) 

•  had  younger  siblings. 1 

No  characteristic  alone  was  statistically  significant  enough  to  predict 
retention.  Therefore,  the  Oaks  study  group  used  combinations  of  these 
characteristics  with  other  information  to  predict  retention.  The  group  used 
stepwise  linear  regression  techniques  to  build  models  to  predict  PTS  for 
each  of  the  21  groups  shown  in  figure  l.2  The  number  of  officers  studied 
in  each  cell  is  indicated.  At  the  time  of  the  study,  some  of  these  officers 
were  still  on  active  duty,  and  others  had  separated  from  the  Air  Force.  The 
officers  studied  were  grouped  into  different  cells  to  isolate  any  year-of-com- 
missi  .  effect  which  might  influence  retention  behavior.  For  example,  t lie 
1964—74  cells  contained  officers  who  may  have  joined  the  Air  Force  to  avoid 
service  in  the  Army  during  the  Vietnam  War.  The  1981-84  cells  contained 
pilots  who  were  still  serving  their  initial  training  commitments  and  were 
therefore  not  eligible  to  voluntarily  separate  from  the  Air  Foree.  Officers 
were  also  grouped  to  isolate  source-of-eommission  effects:  USAFA.  Officer 
Training  School  with  no  prior  enlisted  service  (OTS  NFS).  Officer  Training 
School  with  prior  enlisted  service  (OTS  PRIOR),  and  AFROTC. 

Finally,  the  study  made  a  distinction  between  pilots  and  nonpilots 
because  the  economic  incentives  to  leave  the  Air  Force  are  generally  greater 


1965-1974 

PILOT 


NON¬ 

PILOT 


19.5-1980 

PILOT 


NON¬ 

PILOT 


1981-1984 

PILOT 


NON¬ 

PILOT 


213 

195 

5 

217 

238 

237 

73 

' 

295 

440 

263 

1 13 

465 

476 

280 

203 

530 

OTS 


USAFA 

OTS  NPS 

PRIOR 

AFROTC 

744 

811 

49  j 

867 

767 

735 

231 

962 

Source:  Officer  Selection  “(ran  We  Select  to  Retain?"  (Randolph  AF71.  Tex.:  Officer  Selection  Study  Group. 
Head  quarter*  Air  Training  Command.  1989). 


Figure  1.  Keeper  Study  Officer  Population 
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for  pilots/  The  study  group  considered  some  cells  loo  small  to  analyze  (cells 
not  analyzed  are  indicated  by  shading).  From  a  group  of  about  500  potential 
predictor  variables,  a  computer  program  selected  lor  each  cell  those  vari¬ 
ables  that  best  predicted  actual  officer  retention.  The  study  group  felt  the 
resulting  regression  equations  adequately  modeled  retention  behavior  and 
decided  to  use  the  results  for  the  1975-80  cells  to  estimate  ITS  for  future 
officer  candidates. 

This  study  does  not  critique  the  findings  or  methodology  of  the  Keeper 
Study.  Some  valid  propensity -to-stay  measures  are  assumed,  and  this 
work  in  no  wav  depends  on  the  Keeper  project.  This  study  examines  wavs 
AFROTC  might  implement  PI'S  information  in  its  scholarship  program  and 
estimates  the  impacts,  costs,  and  benefits  of  various  implementation 
options. 


Officer  Candidate  Testing 

The  Air  Force  currently  uses  the  Air  Force  Officer  Qualifying  Test  (AFOQT) 
and  the  Scholastic  Aptitude  Test  (SAT)  to  predict  success  in  training.4  The 
AFOQT  measures  candidate  aptitudes  for  officer  precommissioning  and 
post  commissioning  training.  Various  AFOQT  versions  have  been  used 
since  1951  to  screen  officer  training  applicants.  Two  features  distinguish 
the  Keeper  Study  factors  from  the  AFOQT.  First,  the  Keeper  Study  predicts 
long-term  retention.  The  time  horizon  is  much  greater  than  that  of  the 
AFOQT.  which  predicts  success  in  training.  Second,  much  of  the  Keeper 
Study  predictor  data  are  biographical,  encompassing  information  about 
each  applicant’s  childhood,  family,  motivation,  employment  history,  high 
school  and  college  activities,  life  events,  and  precommissioning  living 
interests.  Although  distinctive,  screening  with  biographical  data  does  have 
a  precedent.  Between  1951  and  1981  the  Air  Force  considered  biographical 
data  to  score  the  AFOQT.  However,  in  1982  the  test’s  biographical  section 
was  deleted.  There  is  no  official  explanation  for  this  change  in  the  AFOQT.  ’ 

The  Leadership  Effectiveness  Assessment  Profile  (LEAP),  a  related  re¬ 
search  effort  being  funded  by  the  Air  Force  Human  Resources  Laboratory, 
aims  to  assess  an  individual's  potential  for  leadership  and  management, 
predisposition  toward  commitment  to  the  Air  Force  as  an  institution,  and 
ability  lo  function  well  in  team  situations.  LEAP  will  also  be  biographical- 
survey  based.  Results  are  expected  to  be  available  for  Air  Force  implemen¬ 
tation  by  1993/’ 


Comments 

The  recent  decision  bv  the  Air  Force  to  increase  the  active  duty  service 
commitment  for  pilot  training  to  10  years  may  reduce  interest  in  PIS 
research.7  This  decision  aims  to  ease  the  pilot  retention  problem.  However. 
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it  seems  that  the  Air  Force  would  always  be  better  served  by  pilots  who  do 
not  feel  indentured.  Hence,  hiring  pilots  with  a  high  PI'S  is  still  a  valid 
objective.  Finally,  while  this  study  focuses  narrowly  on  PI'S,  it  is  also  a 
model  approach  to  determine  the  impacts  of  other  factors  currently  used 
in  Air  Force  officer  selection  systems. 

Chapter  2  provides  background  information  on  AFROTC  selection  fac¬ 
tors.  Understanding  chapters  3  and  4  requires  training  in  simulation, 
statistics,  or  operations  research.  However,  conclusions  to  be  drawn  from 
these  sections  are  clearly  stated. 


Notes 

1.  The  Oaks  special  study  group  reported  its  findings  in  a  report  common)}'  called  the 
“Keeper  Study"  within  ATC.  The  actual  report  title  is  Officer  Selection.  “Can  We  Select  to 
Retain?"  (Randolph  AFB.  Tex.:  Officer  Selection  Study  Group.  Headquarters  ATC.  1989). 

2.  The  regression  technique  is  widely  used  to  find  the  best  mathematical  fit  of  a  response 
variable  to  a  set  of  predictor  variables.  Cause  and  effect  is  not  implied.  However,  (his 
conclusion  is  often  drawn.  Linear  regression  limits  the  relationship  between  response  and 
predictor  to  certain  mathematically  tractable  forms.  Stepwise  linear  regression  selects  from 
a  group  of  possible  predictors  those  that  best  fit  the  response  data.  Stepwise  regression  is 
sometimes  criticized  because  it  relies  on  a  computer  to  search  (without  preconceptions)  for 
a  good  mathematical  relationship.  Critics  argue  that  researchers  ought  to  have  prior 
reasons  to  believe  a  response  is  related  to  a  predictor  variable. 

3.  Enlisted  Air  Force  members  with  college  degrees  may  enter  the  officer  ranks  through 
OTS.  These  individuals  may  retire  after  20  years  of  military  service,  and  to  retire  at  their 
officer  grades,  they  must  have  at  least  10  years  of  commissioned  service.  Officers  with  prior 
enlisted  service  have  higher  retention  rates  in  the  first  10  years  of  commissioned  service 
because  they  are  closer  to  retirement  eligibility. 

4.  The  Air  Force  Officer  Qualifying  Test  (AFOQT)  consists  of  16  subtests  that  are  used 
in  different  combinations  to  form  five  composite  scores:  pilot,  navigator  technical,  academic 
aptitude,  verbal,  and  quantitative.  These  composites  are  used  to  predict  success  in  different 
training  programs. 

5.  One  principal  told  the  author  that  the  change  was  in  response  to  pressure  from  an 
organization  which  felt  the  biographical  portion  of  the  AFOQT  was  discriminatory  against 
blacks. 

6.  Tom  Watson,  “leadership  Effectiveness  Assessment  IVofile."  point  paper  (Brooks  AFB. 
Tex.:  Air  Force  Human  Resources  laboratory,  undated). 

7.  The  Air  Force  requires  that  individuals  sign  an  agreement  to  remain  in  the  Air  Force 
for  a  specified  period  of  time  after  receiving  some  types  of  training.  The  length  of  time  is 
called  the  active  duty  service  commitment  (ADSC).  The  ADSC  ensures  that  the  Air  Force 
realizes  a  fair  return  on  its  training  investments.  In  March  1990  the  Air  Force  announced 
that  the  ADSC  for  pilot  training  would  increase  from  seven  lo  10  years  for  those  entering 
training  after  I  October  1991. 
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Chapter  2 


Current  AFROTC  Scholarship 
Allocation  System 

The  AFROTC  commissioning  program  seeks  io  produce  the  right  numbers 
and  kinds  of  officers  at  the  right  time  within  budget  constraints.  Production 
goals  are  established  by  Headquarters  USAF/DPPP  (Deputy  Chief  of  Staff, 
Personnel.  Directorate  of  Personnel  Programs,  Force  Programs  Division)  in 
consultation  with  all  commissioning  sources.  For  fiscal  year  1992,  the 
AFROTC  production  goal  is  about  2,100  line  and  100  nonline  (e.g.,  doctors, 
nurses,  lawyers)  officers.1  This  chapter  discusses  the  mechanisms 
AFROTC  uses  to  achieve  its  goals.  Before  discussing  these  controls.  1  want 
readers  to  understand  the  standard  AFROTC  course  of  instruction. 


AFROTC  Military  Course  of  Instruction 

Currently,  about  150  colleges  and  universities  have  AFROTC  detach¬ 
ments.  Individuals  earn  an  Air  Force  commission  through  AFROTC  in  one 
of  two  ways.  Figure  2  illustrates  the  usual  AFROTC  life  cycle,  which  is  a 
four-year  sequence  of  instruction  and  evaluation.2 


GENERAL 
MILITARY  COURSE 


PROFESSIONAL 
OFFICER  COURSE 


FRESHMAN 
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e 

SOPHOMORE 

AS  100 
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JL 

AS2C0 

4  OR  6  WEEKS 
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1* 

|| 

SENIOR 

AS  300 

jj 

AS  400 

Figure  2.  AFROTC  Life  Cycle 

The  General  Military  Course  (GMC).  which  essentially  is  open  to  all 
students  who  hope  to  earn  an  Air  Force  commission,  consists  of  (he 
freshman-level  Aerospace  (AS)  100  class  and  the  sophomore-level  AS  200 
class.  Cadets  spend  one  classroom  hour  each  week  on  military  st  udies  and 
one  hour  each  week  on  training.  During  the  summer,  between  the 
sophomore  and  junior  years,  selected  cadet  s  attend  four  weeks  of  basic  Held 
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training  at  an  Air  Force  base.  These  cadets  then  enter  the  lYofessional 
Officer  Course  (POC).  which  is  composed  of  the  junior- level  AS  300  class 
and  the  senior-level  AS  400  class.  Classroom  contact  hours  increase  to 
three  hours  each  week  with  a  fourth  hour  devoted  to  training. 3 

The  second  road  to  an  AFROTC  commission  is  a  two  year  program  that 
consists  of  the  POC.  This  option  requires  each  student  to  attend  two 
additional  weeks  of  summer  training  between  the  sophomore  and  junior 
years  as  a  substitute  for  the  CMC. 


Right  Numbers,  Right  Kinds 

There  may  be  more  cadets  enrolled  in  the  GMC  than  the  Air  Force 
ultimately  will  need.  Consequently,  entry  into  the  POC  (junior  year  of 
instruction)  is  not  automatic:  rather,  it  is  contingent  on  the  Air  Force’s 
forecast  needs  at  the  time  of  graduation.  These  needs  are  stated  in  terms 
of  AFROTC  production  in  the  following  categories: 

•  Pilot  (one-third  to  be  technical  majors) 

•  Engineer  (by  major) 

•  Navigator  (one-third  to  be  technical  majors) 

•  Scientific-Technical  (by  major) 

•  Missile  (one-third  to  be  technical  majors) 

•  Nontechnical 

•  Nonrated  Operations4 

The  mechanism  used  to  control  which  cadets  enter  the  POC  is  (he 
Weighted  Professional  Officer  Course  Selection  System  (WPSS).  This  selec¬ 
tion  process  originates  at  the  AFROTC  detachment  and  initially  assigns 
each  cadet  a  quality  index  score  (Q1S)  in  the  latter  part  of  the  sophomore 
year.  This  score  is  based  on  a  number  of  factors,  including  the  Air  Force 
Officer  Qualifying  Test  (AFOQTj.  physical  qualifications,  and  leadership  and 
military  aptitude  ratings  by  the  detachment  commander  and  other  officers. 
Boards  composed  of  Headquarters  AFROTC  officers  offer  cadets  entry  into 
the  POC  based  on.  among  other  things,  the  quality  index  score,  cadet 
production  category  preferences,  college  grade  point  average  (GPA). 
academic  major,  AFOQT,  and  projected  Air  Force  requirements.  Since  there 
is  a  minimum  quality  index  score  required,  a  cadet  can  be  prevented  from 
entering  the  POC  at  the  detachment  level.  Selectees  may  enter  the  POC 
and  the  Obligated  Reserve  Section  the  first  day  of  (heir  junior  year.  At  (his 
point,  a  cadet  is  obligated  to  complete  college  and  to  begin  four  years  of 
active  duty  upon  graduation.  The  Air  Force  has  a  number  of  recoupment 
options  if  a  cadet  fails  to  complete  school  or  the  AFROTC  program.5 

While  the  WPSS  can  limit  the  number  of  cadets  who  enter  the  POC.  it 
does  not  help  ensure  that  the  minimally  required  numbers  and  kinds  will 
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be  available  for  screening.  Consequently.  AFROTC  uses  two  scholarship 
programs  to  attract  certain  hard-to-fill  production  categories.  In  fiscal  year 
1990  the  Air  Force  will  have  approximately  5,400  cadets  on  scholarship 
(freshmen,  sophomores,  juniors,  and  seniors).  Approximately  one-half  of 
the  officers  who  enter  the  program  through  AFROTC  receive  some  sort  of 
Air  Force  scholarship  to  pay  for  books,  tuition,  and  fees  for  between  two 
and  four  years  (some  seholarship  students  receive  no  funding  for  part  or 
all  of  their  freshman  and  sophomore  years). R 


Four-Year  Scholarship  Program 

Some  Air  Foree  specialties  require  specific  academic  preparation — usual¬ 
ly  an  engineering  or  a  scientific  degree.  Historically,  the  Air  Force  Academy 
does  not  produce  enough  technical  officers  to  meet  Air  Force  needs,  and 
the  OTS  does  not  attract  sufficient  high-quality,  technically  educated 
individuals.7  To  fill  this  void.  AFROTC  offers  four-year  scholarships  to 
high-quality  students  to  study  specific  disciplines.  Approximately  15.000 
individuals  apply  for  the  four-year  scholarships  each  year.  The  Air  Force 
offers  scholarships  to  about  2.400  applicants,  and  900  to  1 .200  individuals 
actually  accept  the  scholarships  and  enter  the  freshman  class.  About  99 
percent  of  these  scholarships  are  awarded  to  technical  majors.8 

Selection  Process  Timing  and  Criteria 

Four-year  scholarships  are  typically  offered  to  high  school  seniors. 
AFROTC  accepts  applications  from  June  through  November  for  college 
entries  the  following  September.  Each  applicant  is  rank  ordered  by  a  board 
of  officers  assigned  to  AFROTC  detachments.  About  one-third  of  an 
applicant’s  score  is  based  on  high  school  studies.  Specific  items  of  interest 
are  Scholastic  Aptitude  Test  (SAT)  scores  (minimum  1.000  total.  500 
mathematics,  450  verbal).  American  College  Test  (ACT)  scores  (minimum 
23  composite,  20  mathematics,  19  English),  grade  point  average  (minimum 
2.5  on  a  4.0  scale),  class  rank  (minimum  top  25  percent),  and  classes  taken. 
Together,  these  Items  indicate  an  applicant's  ability  to  complete  college  level 
work.  About  one-third  of  an  applicant’s  board  score  is  based  on  extracur¬ 
ricular  activities  in  high  school  to  get  a  measure  of  leadership  potential  as 
an  officer.  The  final  one-third  of  the  score  is  based  on  appraisals  of  the 
applicant  by  an  Air  Force  officer  and  high  school  teachers  and  counselors.9 

Students  apply  and  compete  for  scholarships  according  to  academic 
discipline.  Competition  is  keen  in  some  disciplines.  In  others,  most 
applicants  who  meet  minimum  standards  are  offered  a  scholarship.  Those 
receiving  four-year  scholarships  are  classified  into  a  production  category  in 
the  latter  part  of  their  sophomore  year  through  the  WPSS.  Figure  3 
illustrates  the  life  cycle  of  four-year  scholarship  recipients. 
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Figure  3.  AFROTC  Student  Flows 


Scholarship  Types 

There  are  four  major  award  categories  in  the  Four-Year  Scholarship 
Program  (FYSP).  If  there  is  an  AFROTC  detachment  at  a  university,  and  a 
Type  J  four-year  scholarship  recipient  can  get  accepted  at  that  school,  the 
Air  Force  pays  for  books,  tuition,  and  fees,  regardless  of  the  cost.  Type  II 
four-year  scholarships  pay  up  to  a  dollar  ceiling.  For  the  1990-91  school 
year,  the  Air  Force  plans  for  Type  I  awards  to  comprise  about  20  percent  of 
the  FYSP.  The  annual  Type  II  ceiling  will  be  $8,000.  For  those  with  suspect 
academic  ability.  AFROTC  offers  Type  II  three-and-one-half-year  guaran¬ 
teed  and  three -year  guaranteed  scholarships.  These  scholarships  are 
awarded  after  the  student  exceeds  a  specified  minimum  academic  perfor¬ 
mance  in  college  for  either  one  semester  (for  the  three-and-one-half-year 
scholarship)  or  two  semesters  (for  the  three-year  scholarship). 10 

Student  Obligations 

Students  who  receive  financial  aid  under  this  program  enlist  in  the 
Obligated  Reserve  Section  at  the  start  of  their  sophomore  year.  These 
cadets  agree  to  finish  school  and  serve  four  years  of  active  duty  or  be  subject 
to  Air  Force  recoupment  action. 1 1 
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Uncertainties 


Several  factors  complicate  FYSP  management.  When  these  scholarships 
are  offered,  the  AFROTC  objective  is  to  produce  required  numbers  by 
category  four  years  hence.  One  uncertainty  is  that  historically  only  about 
50  percent  of  awardees  ultimately  will  accept  their  scholarships.  AFROTC 
is  in  direct  competition  with  the  Air  Force  Academy,  other  military  services, 

in 

and  other  funding  sources  for  these  students.  Therefore,  it  is  not  likely 
that  the  desired  number  of  students  by  academic  major  will  accept  scholar¬ 
ships.  Some  majors  may  be  overrepresented,  while  others  may  be  under¬ 
represented.  A  second  source  of  uncertainty  is  that  some  scholarship 
recipients  do  not  know  which  university  they  will  attend.  This  uncertainty 
complicates  budget  management  and  also  complicates  predicting  proh 
ability  of  graduation,  as  schools  have  different  attrition  rates.  Hence, 
meeting  production  goals  within  cost  constraints  with  the  FYSF  alone  is 
difficult.  A  final  problem  is  that  production  requirements  for  four  years  in 
the  future  are  only  estimates  and  will  likely  change.  To  deal  with  the 
uncertainty  inherent  in  the  FYSP,  AFROTC  uses  the  College  Scholarship 
Program  (CSP)  to  make  midcourse  adjustments. 13 

College  Scholarship  Program 

The  CSP  augments  the  four-year  scholarship  program  to  produce  officers 
subject  to  numbers,  types,  budgets,  and  other  constraints.  Figure  3 
illustrates  that  the  CSP  makes  awards  to  individuals  who  are  already  in 
college.  Roughly  50  percent  of  AFROTC  scholarships  fall  into  this  category, 
with  about  1 .000  awards  given  to  2,000  applicants  annually. 14 

The  CSP  has  several  attractive  features.  This  program  has  less  uncer¬ 
tainty  than  the  FYSP.  Since  the  applicant's  school  and  major  are  known, 
costs  are  more  predictable.  There  is  a  student  track  record  of  collegiate 
academic  performance.  Since  awards  are  for  less  than  four  years,  the  CSP 
costs  less  on  a  per  capita  basis.  Finally,  the  Air  Force  has  a  better  estimate 
of  requirements,  as  students  are  closer  to  graduation.  These  advantages 
lead  one  to  believe  that  the  Air  Force  should  place  all  its  resources  in  the 
CSP.  However,  the  CSP  does  not  make  100  percent  of  the  AFROTC  awards, 
because  it  is  not  sufficiently  attractive  to  students  in  some  technical  majors 
or  to  students  attending  the  highest  quality  schools  (as  measured  bv  cost). 
As  these  individuals  are  already  in  college,  they  have  found  some  form  of 
financial  aid  and  are  closer  to  realizing  starting  civilian  salaries  which 
exceed  a  second  lieutenant’s  pay.  Consequently,  the  Air  Force  is  not  as 
attractive  financially. 15 

Selection  Process 

College  scholarship  applicants  are  rank  ordered  through  a  board  process. 
About  one-fourth  of  an  applicant's  score  is  awarded  for  academic  potential 
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as  measured  by  the  SAT,  ACT,  or  academic  aptitude  score  on  the  AFOQT. 
About  one-third  of  the  score  is  based  on  academic  performance  as  measured 
by  college  GPA  and  course  difficulty.  The  remainder  of  the  score  is  based 
on  AFROTC  Form  36.  AFROTC  Scholarship  Nomination.  This  form  con¬ 
tains  an  applicant’s  appraisal  based  on  an  interview  with  an  AFROTC 
officer,  an  applicant’s  statement  of  desire  for  a  commission,  a  list  of 
extracurricular  activities,  previous  employment,  academic  major,  race,  and 
sex.  Unlike  the  FYSP.  college  scholarships  are  awarded  bv  production 
category  (e.g.,  pilot,  navigator).  Although  85  percent  of  the  CSP  recipients 
have  technical  majors,  prerequisite  academic  majors  exist  only  for  engineer¬ 
ing  and  scientific-technical  production  categories."’ 

Obligation 

Recipients  of  financial  assistance  under  the  CSP  enter  the  Obligated 
Reserve  Section  at  the  time  financial  aid  begins  or  on  the  first  day  of  their 
sophomore  year,  whichever  is  later. 17 


Right  Time 

Graduates  from  the  Air  Force  Academy  and  OTS  enter  aetive  duty  upon 
graduation.  This  is  not  the  case  for  AFROTC  graduates.  The  Air  Force  may 
wait  up  to  12  months  after  graduation  before  bringing  AFROTC  graduates 
on  active  duty  (if  at  all).18  This  gives  the  Air  Force  latitude  in  two  respects. 
First,  since  college  graduation  dates  do  not  align  with  the  start  of  the 
government’s  fiscal  year,  AFROTC  graduates  can  be  brought  on  active  duty 
in  one  of  two  fiscal  years.  This  gives  the  Air  Force  flexibility  in  meeting 
end-of-year  strength  objectives  (end  strength).  Second.  AFROTC  graduates 
give  Air  Force  programmers  flexibility  in  meet  ing  Military  Personnel  Account 
(MPA)  constraints.  An  individual  who  comes  on  to  active  duty  on  1  August 
will  have  to  be  paid  for  two  months  in  that  fiscal  year  and  will  have  to  be 
counted  against  end  strength.  However,  if  the  Air  Force  waits  until  1 
October  to  bring  the  individual  on  active  duty,  the  AFROTC  graduate  will 
not  count  or  cost  in  the  previous  fiscal  year's  budget.  The  difference 
between  bringing  all  AFROTC  cadets  on  active  duty  at  graduation  as 
opposed  to  12  months  later  is  about  a  one-time  $50 -million  decision: 

2,500  2Lt/year  x  $20.000/2Lt/year  =  $50M. 

Only  a  portion  of  this  could  be  saved  by  adjusting  accession  timing. 


Right  Cost — AFROTC  Scholarship  Budget 

The  AFROTC  scholarship  budget  is  calculated  from  the  current  year’s 
average  cost  per  scholarship  (adjusted  tor  inflation)  multiplied  times  the 
number  of  scholarships  awarded.  For  academic  year  1 988-89.  the  average 
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cost  per  student  was  $5,559. 19  Under  certain  conditions,  this  approach  is 
statistically  sound  because  there  are  large  numbers  of  scholarships 
awarded  (5.400  in  effect  for  academic  year  1989-90).  When  large  numbers 
are  involved,  actual  average  costs  are  likely  to  be  near  historical  average 
costs.  This  approach  requires  few  changes  in  the  scholarship  programs 
from  one  year  to  the  next.  Students  must  attend  schools  in  the  same  tuition 
category.  All  schools  must  inllate  their  costs  similarly.  Ttie  dollar  ceiling 
on  low-cost  schools  must  change  at  the  inflation  rate.  The  ratio  of  students 
in  high-cost  and  low-cost  schools  must  remain  constant.  To  the  extent  that 
these  tilings  do  not  happen,  the  AFROTC  scholarship  budget  will  not  match 
expenditures.  (Chapter  4  contains  estimates  on  how  much  outlays  can  be 
expected  to  differ  from  the  budget.) 


Other  Constraints 

In  addition  to  meeting  production  goals,  other  potential  AFROdC  con¬ 
siderations  might  be  managed  through  the  scholarship  program.  If  there 
were  any  particular  group  the  Air  Force  could  not  gel  in  sufficient  numbers, 
scholarships  might  be  used  as  an  incentive.  For  example,  scholarships 
might  be  used  to  attract  blacks,  hispanics.  and  white  females.  Another 
issue  is  detachment  viability.  By  law,  if  an  AFROTC  unit  has  fewer  than 
17  cadets  in  the  junior  class  for  four  consecutive  years,  the  detachment 
must  be  closed.  The  scholarship  program  could  be  used  to  manage 
detachment  viability. 

Propensity-to-Stay  Considerations 

Current  AFROTC  policy  is  to  award  scholarships  to  applicants  with  the 
highest  input  quality.  No  study  has  determined  the  validity  of  these 
measures  to  predict  success  as  an  officer,  although  some  of  t  hem  do  predict 
success  in  training.  As  discussed  in  chapter  1.  the  Keeper  Study  reports 
success  in  predicting  retention.  Propensity  to  stay  could  be  incorporated 
into  the  AFROTC  WPSS  or  scholarship  award  processes  if  there  were 
chronic  retention  problems  with  a  particular  group. 


Remarks 

'Hie  primary  goal  of  the  AFROTC  commissioning  program  is  to  produce 
the  right  numbers  and  kinds  of  officers  at  the  right  time  within  budget 
constraints.  The  Four-Year  Scholarship  Program  attracts  desired  academic 
majors  to  AFROTC.  The  principal  shortcoming  of  the  four-year  program  is 
its  uncertainty.  The  College  Scholarship  Program  allows  midcour.se  adjust - 
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merits.  AFROTC  uses  the  Weighted  Professional  Officer  Course  Selection 
System  to  make  final  adjustments  before  cadets  enter  their  junior  year.  At 
the  present  time,  exercising  these  three  controls  is  a  complex,  labor-inten¬ 
sive  process.  The  next  chapter  presents  an  approach  that  could  aid 
AFROTC  managers  in  system  control. 
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Chapter  3 


Integrating  Propensity  to  Stay  into 
the  AFROTC  Commissioning  Program 


The  Air  Fotc  dors  not  want  all  officer  accessions  to  stay  in  the  service 
until  retirement. 1  Any  commissioning  effort  to  alter  retention  rates,  either 
higher  or  lower,  must  consider  projected  manning  in  each  production 
category.  With  this  caveat.  AFROTC  could  incorporate  propensity  to  stay 
(FI'S)  information  at  three  decision  points:  when  awarding  four-year 
scholarships  (FYSP),  when  awarding  scholarships  to  those  already  enrolled 
in  College  Scholarship  Program  (CSP).  and  when  screening  applicants  for 
the  Weighted  Professional  Officer  Course  Selection  System  (WPSS).  The 
PTS  information  is  useful  only  when  true  decisions  are  made.  If  there  were 
too  few  applicants  for  a  scholarship  and  AFROTC  selected  all  applicants 
meeting  minimum  qualifications.  PI'S  probably  would  not  change  any 
decisions.  However,  when  applications  exceed  allocations,  PI'S  could  make 
a  difference. 


Implementation  Options 

For  each  of  the  three  decision  points.  AFROTC  has  at  least  four  implemen¬ 
tation  options,  ranging  from  a  slight  alteration  to  an  extensive  revision  of 
current  scholarship  applicant  selection  procedures. 

Option  One — Objective 

Under  this  option.  PI'S  impacts  would  be  completely  objective.  Hoard 
members,  who  today  subjectively  rate  applicants,  would  not  see  raw  PI'S 
information.  PI'S  would  influence  allocations  in  a  wav  whic  h  would  be 
transparent  to  decision  makers,  either  by  embedding  the  information  in 
another  measure  or  by  objectively  adding  points  to  an  applic  ant  s  board 
score.  The  main  advantage  this  approach  offers  is  that  PI'S  impac  ts  could 
he  both  estimated  and  controlled.  At  the  present  time.  AFROTC  does  not 
employ  an  indicator  in  a  completely  objective  manner. 

Option  Two — Subjective 

I  Fnder  this  option,  decision  makers  would  have  access  to  each  applicant's 
ITS  information  and.  within  wide  bounds,  act  on  it  subjectively.  This  is 
the  option  board  members  use  today  for  most  other  quality  indicators. 
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Option  Three — Combination 

Option  three  is  a  combination  of  options  one  and  two.  It  shows  Id'S 
information  to  board  members  and  also  requires  an  objective  impact.  This 
option  ensures  that  PI'S  lias  some  minimum  impact  while  allowing  extreme 
scores  to  carry  disproportional  weight  in  the  allocation  decision,  AFROTC 
currently  handles  a  lew  indicators  in  this  manner. 

Option  Four — Optimization 

The  first  three  options  require  little  modification  to  current  AFROTC 
selection  prexesses.  Option  four  is  different.  It  is  an  optimization  option 
that  not  only  allows  PTS  to  influence  individual  allocations,  it  also  affords 
AFROTC  other  management  opportunities.  This  option  requires  boards  to 
divide  an  applicant  pool  into  three  groups.  Individuals  who  were  so 
outstanding  by  current  standards  that  they  would  receive  an  allocation  no 
matter  what  their  ITS  might  be  would  comprise  group  one.  Group  three 
would  contain  people  of  such  low  quality  bv  current  standards  that  they 
would  not  receive  an  allocation  no  matter  what  their  PI'S.  Group  two  would 
be  "gray-zone"  individuals.  While  some  would  be  better  than  others  by 
current  measures,  the  differences  would  not  be  so  great  that  other  factors 
should  not  influence  the  allocation  decision.  The  objective  would  be  to 
select  from  the  gray  zone  a  subset  that  had  higher  FI'Ss.  The  selection 
would  also  simultaneously  satisfy  other  AFROTC  constraints,  such  as  total 
production  goals,  budget  limits,  minority  production  goals,  engineering 
production  goals,  or  detachment  viability. 

Weighted  Professional  Officer 
Course  Selection  System 

Suppose  the  Air  Force  has  determined  that  pilot  retention  is  too  low  and 
should  be  altered  by  training  different  types  of  people.  At  the  present  time. 
AFROTC  pilot  allocations  are  made  through  the  WPSS.  When  making  these 
allocations.  WF’SS  boards  primarily  consider  two  measures:  a  cadet  s 
quality  index  score  (QIS)  and  the  AFOQT  pilot  component.2  Figure  4  shows 
pilot  allocution  award  probability  as  a  function  of  QIS  and  pilot  AFOQT. 
t  hese  data  are  for  nonsebolarship  cadets  considered  in  fiscal  year  1 989  lor 
fiscal  year  1991  college  graduation.  All  data  are  for  applicants  with  pilot 
as  their  first  preference.  Small  cells  are  not  displayed.  For  example.  56 
cadets  appeared  with  a  QIS  in  the  80-89  range  and  with  a  pilot  score  in  the 
71  -80  range.  Of  these.  44.  or  79  percent,  received  pilot  alloc  ations. 

Figure'  4  reveals  several  tilings.  First,  if  the  pilot  score  is  held  constant, 
pilot  allocation  probability  generally  increases  with  higher  QIS  (reading 
Iroin  left  to  right  within  a  row).  Second,  if  QIS  is  held  constant,  allocation 
probability  increases  with  higher  pilot  score  (reading  from  top  to  bottom  in 
a  column).  As  would  be-  expected  when  dealing  with  data,  exceptions  exist 
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Figure  4  Pilot  Allocation  Probabilities 


to  these  generalizations.  A  third  observation  is  that  the  highest  allocation 
probability  seems  to  he  in  t  he  .8  to  .9  range.  Wit  bin  a  c  ell  in  figure  4.  where 
all  applicants  have  about  the  same  QIS  and  pilot  score,  how  is  it  that  some 
are  selected?  The  answer  is  generally  found  within  the'  QIS.  whic  h  is  a 
composite  of  other  measures.  If  any  one  component  is  strikingly  low.  a 
cadet  might  not  get  an  allocation.  For  example,  a  low  CPA  might  cause 
rejection  (see  chapter  2  for  QIS  elements). 

WI’SS  boards  begin  their  work  in  February.  All  a  lineal  ions  are  not  usually 
made  during  the  first  board  meeting.  Rather,  the  proc  ess  is  iterative.  Hascd 
on  Air  Forc  e  requirements  and  individual  preferences  and  qualifications, 
AFROTC  oilers  some  applicants  alien  ations.  Not  all  cadets  are  ottered  their 
first  choice.  For  example,  electrical  engineers  on  AFROTC  sc  holarships 
usually  won't  receive  a  first-choice  pilot  allocation.  They  normally  receive 
electrical  engineering  allocations.  As  requirements  c  hange,  and  cadets 
accept  or  reject  WPSS  alloc  ations,  additional  boards  may  be  convened  oven 
a  period  of  months.  Cadets  not  offered  an  allocation  on  one  board  are 
c  arried  over  and  reconsidered  on  subsequent  boards.  1 

How  can  FI'S  information  be  Integrated  into  the  WI’SS?  Four  general 
ways  already  have  been  mentioned  to  integrate*  FIS  into  the*  allocation 
process.  Fac  h  option  is  now  discussed  in  detail. 


Option  One — Objective 

PTS  could  be  incorporated  into  QIS  without  showing  the  PTS  component 
to  board  members.  Currently,  the  maximum  QIS  is  124.  If  PTS  were 
incorporated  into  a  QIS  with  a  10-percent  weight,  then  12.4  would  be  the 
maximum  PTS  points. 

Figure  5  gives  some  insight  into  one  PTS  point -awarding  subtlety.  Listed 
along  each  matrix  top  are  some  possible  original  QISs  that  range  from  zero 
to  124.  Along  the  left  side  of  the  matrix  are  some  possible  PTS  awards.  The 
matrix  entries  reflect  the  change  from  the  original  QIS  that  would  result 
from  awarding  PTS  points  under  two  different  methods.  Under  method  one, 
the  maximum  QIS  would  remain  124,  with  up  to  12.4  points  awarded  for 
PI’S.  In  method  two,  up  to  14  points  would  be  added  to  current  QISs: 
14/(124  +  14)  =  10  percent.  The  two  methods  produce  different  results. 
For  example,  under  method  one,  an  individual  with  an  original  QIS  of  85, 
who  received  6.2  PTS  points,  would  see  a  -2.3  change  in  the  QIS  for  a  new 
QIS  of  82.7.  Under  this  implementation,  one  conclusion  springs  forth 
immediately:  PI'S  points  tend  to  help  those  with  low  original  QISs  and  hurt 
those  with  high  QISs  (read  left  to  right  within  a  row).  Reading  within  a 
column,  the  difference  between  awarding  zero  and  12.4  points  is  always 
12.4.  Thus,  an  original  QIS  of  65  would  transform  into  the  range  (65  -  6.5, 
65  +  5.9)  =  (58.5,  70.9)  and  (70.9  -  58.5  =  12.4).  The  most  change  an 
individual  would  see  in  the  QIS  would  be  12.4.  Depending  on  where  an 
individual  was  in  figure  4,  +.05  points  would  be  about  the  probability  that 
receiving  a  pilot  allocation  would  change  (where  1.00  is  selection  with 
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Figure  5.  PTS-Induced  Change  in  QIS 
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certainty  and  0.00  is  nonselection).  Method  two  would  produce  about  the 
same  change  of  magnitude  except  that  the  FTS  impact  would  be  inde¬ 
pendent  of  the  original  QIS.  Hence,  under  option  one,  PTS  would  have  only 
a  marginal  effect. 

An  issue  that  must  be  addressed  is  the  propensity-to-stay  point  distribu¬ 
tion  plan  as  a  function  of  PTS.  If  screening  out  individuals  with  a  low  PTS 
is  the  goal,  AFROTC  could  achieve  maximum  effect  by  awarding  zero  points 
for  a  low  PTS  and  nearly  14  points  (under  method  two)  for  all  other  PTSs. 
Likewise,  maximum  advantage  could  be  given  to  those  with  a  high  PTS  by 
awarding  them  14  points  while  giving  everyone  else  zero  points.  A  more 
traditional  distribution  would  award  seven  points  for  a  midrange  PTS  and 
zero  or  14  points  for  PTS  extremes.  It  may  also  be  the  case  that  different 
distributions  should  be  used  for  different  production  categories.  For  ex¬ 
ample.  if  there  were  a  production  category  with  retention  rates  that  were 
too  high,  a  low  PTS  might  be  desirable. 

Option  Two — Subjective 

A  PTS  measure  could  be  shown  to  board  members  without  being  incor¬ 
porated  into  the  QIS.  Requiring  no  change  to  current  QIS  calculations,  this 
option  would  be  easy  to  implement.  Board  members  could  be  advised  about 
the  appropriate  PTS  weights.  However,  this  option  produces  higher  uncer¬ 
tainty  about  PTS  impacts.  Some  board  members  may  reject  PTS  validity 
and  place  no  weight  on  it.  Others  might  take  the  opposite  extreme.  One 
may  speculate  that  under  this  option.  PTS  would  mirror  other  QIS  com¬ 
ponents  that  are  available  to  the  board.  A  particular  component  may  be 
heavily  weighted,  particularly  if  extremely  low.  For  example,  suppose  two 
cadets  have  exactly  the  same  pilot  score  and  QIS.  One  has  a  low  GPA,  which 
is  compensated  for  by  a  detachment  commander's  high  rating.  The  other 
has  nominal  QIS  components.  The  cadet  with  the  low  GPA  is  less  likely  to 
receive  an  allocation.  In  this  example,  a  low  GPA  drove  the  allocation 
decision.  Similarly,  extreme  PTSs  might  carry  great  weight,  all  other  things 
being  equal.  If  the  Air  Force  wants  to  improve  retention  in  a  production 
category,  a  low  PTS  might  cause  rejection  under  the  WPSS.  Ironically,  while 
AFROTC  might  be  less  likely  to  offer  a  pilot  allocation  to  a  cadet  with  a  low 
PTS,  it  might  be  more  likely  to  offer  the  same  cadet  an  allocation  in  a 
production  category  with  retention  considered  too  high.  Hence,  under 
option  two,  PTS  potentially  has  more  impact  than  option  one  for  those  with 
extreme  scores. 

Option  Three — Combination 

Propensity  to  stay  could  be  treated  like  most  other  quality  index  score 
components.  It  could  both  contribute  to  QIS  and  be  available  to  the  board. 
This  option  would  not  eliminate  the  potential  for  great  weight  on  extreme 
scores.  However,  it  would  ensure  that  PTS  carried  some  weight.  As  in 
option  one,  some  adjustment  to  current  procedures  would  be  required. 
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Option  Four — Optimization 

This  option  would  require  board  members  to  separate  applicants  into 
three  groups  without  considering  PI'S.  Group  one  would  be  high-quality 
applicants  who  would  receive  an  allocation  regardless  of  their  PTS.  Group 
two  would  be  gray-zone,  intermediate-quality  applicants,  any  of  whom 
would  be  acceptable.  Group  three  would  be  unacceptable  applicants, 
regardless  of  their  PTS.  Assuming  that  group  one’s  size  is  less  than  total 
allocations,  then  a  group  two  subset  could  be  selected  using  classic 
operations  research  optimization  techniques.  This  case  is  a  zero-to-one 
integer  programming  problem  (again  WPSS  pilot  allocations).  The  following 
equations  define  requirements  and  objectives  for  pilot  production.  These 
equations  do  not  apply  to  other  production  categories. 

Objective:  To  Maximize 
Z|  Pai  *  P4i  *  PTSi  *  Xi 

where  E,  means  the  sum  for  all  gray- zone  pilot  applicants. 

p3)  means  the  probability  that  applicant  i  will  complete  the 
third  (junior)  year.  Ideally,  this  probability  or  persistence 
would  be  estimated  at  least  as  a  function  of  academic  major 
and  school. 

p4i  is  the  fourth  (senior)  year  completion  probability. 

PTS,  means  the  propensity  to  stay  in  years  for  gray-zone  ap¬ 
plicant  i  (PTS  points  here  correspond  directly  to  PTS). 

x,  is  a  decision  variable  (representing  applicant  i)  which  an 
optimization  program  will  set  equal  to  zero  if  no  allocation 
or  one  for  an  allocation. 

*  means  multiply.  I'll  us.  PTS,*x,  equals  PI'S,  or  zero. 

The  following  constraints  govern  pilot  allocation: 

Production 

E,  p3,  *  p41  *  x,  >  pilot  production  goal  for  gray-zone  applicants. 

Minority 

£j  p3j  *  p4J  *  Xj  >  minority  pilot  goal  for  gray-zone  applicants  where 
Index  j  denotes  minorities  (j’s  are  a  subset  of  i  s). 


Engineer 

Pak  *  P4k  *  -  pilot -engineer  goal  for  gray-zone  applicants 

where  index  k  denotes  engineers  (k's  are  a  sub¬ 
set  of  is). 

Budget 

L,  c3i  *  x,  <  gray-zone  pilot  budget  share  junior  year 

where  c31  means  the  cost  that  applicant  i  will  incur  in  the  third 
(junior)  year.  Ideally,  this  cost  would  be  at  least  a  func¬ 
tion  of  school  and  inflation. 

X,  c4,  *  p31  *  xi  <  gray-zone  pilot  budget  share  senior  year 

where  c4i  means  the  cost  that  applicant  i  will  incur  in  the  fourth 
(senior)  year. 


Example 

Suppose  a  WPSS  board  met  and  split  a  pilot  applicant  pool  into  three 
parts.  Table  1  lists  group  two  or  gray-zone  characteristics.  (All  data  are 
notional.  The  PTS  points  for  this  example  simply  reflect  the  expected  years 
the  individual  will  remain  in  the  Air  Force.)  Suppose  that  AFROTC  wanted 
to  satisfy  the  following  conditions: 


Production 

Race 

Engineers 

Budget3 

Budget4 


>  3.5  (expected  production  of  at  least  3.5  people) 

>  1.0  (produce  at  least  one  minority) 

>  2.0  (produce  at  least  two  engineers) 

<  $30,000  (spend  less  than  $30K  in  the  junior  year) 

<  $30,000  (spend  less  than  $30K  in  the  senior  year). 


How  should  AFROTC  select  applicants  to  satisfy  these  conditions  while  it 
maximizes  total  PTS? 

TABLE  1 


Notional  Data 


PILOT 

PERSIST 

COST 

PERSIST 

COST 

NAME 

OIS 

AFOOT 

YEARj 

YEARs 

YEARt 

year4 

MINORITY 

ENGINEER 

PTS 

A 

72 

68 

98 

$  8100 

99 

$  8500 

12  6 

B* 

88 

56 

.88 

4200 

90 

4300 

YES 

10  7 

C* 

70 

59 

86 

5600 

88 

5900 

YES 

YES 

99 

D* 

62 

78 

98 

5100 

98 

5300 

9  8 

19 


TABLE  1  (cont'd) 


PILOT 

PERSIST 

COST 

PERSIST 

COST 

NAME 

O/S 

AFOOT 

YEARj 

YEARj 

YEAFU 

YEAFU 

MINORITY 

ENGINEER 

PTS 

E 

75 

56 

.94 

$15000 

96 

$16000 

YES 

94 

F 

82 

49 

85 

5800 

86 

6100 

YES 

90 

G" 

77 

61 

93 

4700 

.94 

5000 

8.7 

H 

88 

56 

85 

5400 

.90 

5500 

YES 

YES 

86 

I* 

69 

65 

84 

3000 

86 

3100 

YES 

8  6 

J* 

70 

70 

94 

4300 

.97 

4500 

85 

K 

87 

50 

88 

7200 

91 

7600 

YES 

8  2 

L 

79 

70 

.88 

12000 

90 

12700 

YES 

8  0 

M 

73 

54 

83 

8100 

85 

8300 

YES 

7  8 

N 

86 

52 

92 

5000 

94 

5350 

7.5 

O 

62 

78 

96 

5600 

.97 

7000 

YES 

7.3 

P 

89 

52 

.80 

8100 

85 

8500 

YES 

7  1 

Q* 

72 

60 

.98 

3000 

98 

3100 

YES 

7  0 

R 

72 

70 

95 

4300 

.96 

4500 

70 

S 

65 

80 

85 

8100 

.90 

8500 

YES 

7  0 

T 

87 

57 

.85 

15000 

88 

16000 

YES 

YES 

70 

If  AFROTC  selected  B,  C.  D,  G.  I,  J,  and  Q,  the  following  would  be  expected: 


Production 

Race 

Engineer 

Budget^ 

Budget4 

Total  PTS 


=  6.0 
=  2.5 
=  2.3 
=  $29,900 
=  $28,570 

=  53.7 


A  zero-to-one  integer  programming  optimization  code  made  these  selec¬ 
tions.  In  general,  it  selected  those  with  higher  FTSs.  An  individual  not 
selected  generally  had  a  high  cost  or  a  low  FI'S.  Selecting  any  other 
applicant  set  would  result  in  a  violation  of  a  constraint  or  a  lower  total  PTS. 
As  the  solution  indicates,  the  original  budget  constraints  are  not  consistent 
with  production  goals.  AFROTC  would  probably  want  to  impose  additional 
constraints.  For  example,  some  minimum  production  from  high-cost 
schools  might  be  desired.  Maximum  production  levels  should  probably  be 
set.  School  viability  might  be  a  concern.  It  is  possible  to  overconstrain  the 
problem  so  that  no  solution  is  possible.  This  is  also  useful  information 
since  impossible  goals  ought  not  be  sought. 

This  problem  formulation  was  kept  small  and  simple  for  illustration 
purposes.  Some  complexities  have  been  ignored.  This  problem  could 
probably  be  solved  manually.  However,  as  constraint  and  applicant  num¬ 
bers  grow,  computer  assistance  becomes  essential. 

Option  four  gives  great  weight  to  PI'S  for  those  in  the  gray  zone.  It  is  a 
major  modification  to  the  current  system.  The  main  advantage  this  ap¬ 
proach  offers  is  that  it  permits  AFROTC  to  consider  PTS  while  it  satisfies 
other  necessary  constraints.  The  disadvantage  is  that  some  work  would 
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have  to  be  done  to  write  software  and  to  implement  the  new  system. 
Chapter  4  has  additional  information  on  the  implementation  of  option  four. 


College  Scholarship  Program 

A  panel  of  three  professors  of  aerospace  studies  (PAS)  reviews  each 
College  Scholarship  Program  application.  (A  PAS  is  an  Air  Force  colonel  or 
lieutenant  colonel  who  commands  an  AFROTC  detachment.)  Each  officer 
subjectively  assigns  each  record  a  score  of  from  zero  to  100.  Each  member 
assigns  up  to  24  points  to  each  record  to  indicate  academic  potential.  The 
AFOQT  academic  aptitude  component  score  and/or  ACT/SAT  scores  in¬ 
fluence  these  points.  Up  to  33  points  are  awarded  for  academic  perfor¬ 
mance.  Board  members  consider  college  GPA.  AFROTC  grades,  and  course 
difficulty.  The  final  43  points  are  based  on  AFROTC  Form  36.  which 
contains  a  number  of  data  fields  and  personal  comments  including  the  Q1S 
and  the  AFOQT  scores.  Board  members  may  weigh  each  Form  36  item  as 
they  wish,  although  historically  the  PAS  and  applicant  comments  are  the 
most  important.  The  board  members'  scores  yield  a  maximum  possible 
score  of  300.  Boards  are  held  three  times  during  the  year,  depending  on 
the  scholarship  type.  Figure  6  illustrates  the  frequency  with  which  t lie 
February  1989  board  awarded  various  scores.4  For  example,  88  applicants 
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Figure  6.  February  1989  CSP  Board  Results 
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received  a  score  in  the  191-200  range.  The  relatively  high  density  of  scores 
in  the  200-250  range  has  important  implications  for  various  PI'S  implemen¬ 
tation  option  impacts.  This  importance  stems  from  the  AFROTC  policy  of 
awarding  scholarships  based  on  board  rank.  When  awarding  50  scholar¬ 
ships.  AFROTC  selects  the  50  top-ranking  applicants.  For  the  February 
1989  board,  a  small  change  in  score  for  an  individual  in  the  200-250  range 
would  change  relative  rank  much  more  than  the  same  change  in  score  for 
an  individual  in  the  291-300  range. 

Option  One — Objective 

Under  this  option.  PTS  points  would  be  objectively  added  to  board  scores 
without  showing  the  PTS  component  to  board  members.  Using  the  method 
two  approach  discussed  previously.  AFROTC  could  add  up  to  33  PTS  points 
to  an  applicant’s  board  score — 33/(300  +  33)  =  10  percent.  It  could 
distribute  these  PTS  points  in  many  ways.  Each  distribution  method  would 
produce  different  results.  AFROTC  must  anticipate  these  impacts  before  it 
selects  a  distribution  plan.  Figure  7  illustrates  four  potential  plans. 

The  triangular  distribution  reflects  one  way  raw  PTS  information  could 
be  transformed  into  points.  The  average  applicant  would  receive  one-hall' 
the  maximum  possible  PTS  points  (33/2  in  this  case).  Few  applicants 
would  have  very  high  or  very  low  PTS  points.  Finally,  the  highest  FI'S  score 
frequency  would  be  near  the  midrange. 

Under  the  uniform  distribution  plan,  AFROTC  would  give  the  average 
applicant  one-half  the  maximum  possible  PTS  points.  In  this  case,  score 
frequency  would  be  the  same  over  the  entire  range.  There  would  be  as  many 
high  scores  as  low  and  as  midrange. 

The  “Help  High  PTS"  distribution  would  award  maximum  FI'S  points  to 
those  with  the  top  25  percent  raw  scores  and  no  points  to  all  others.  Hence, 
PTS  points  would  help  only  those  with  the  highest  PTSs. 

DENSITY 


Figure  7.  PTS  Distribution  Ptans 


22 


The  “Hurt  Low  PTS"  distribution  would  award  maximum  points  to  the  top 
75  percent  of  the  raw  scores.  The  lower  25  percent  would  receive  no  points 
and  thus  would  be  hurt  relative  to  the  others. 

This  study  used  Monte  Carlo  simulation  to  estimate  the  impact  of 
awarding  PTS  points  by  these  four  distributions.  The  simulation  process 
consisted  of  the  following  sequence: 

1.  Randomly  select  a  sample  of  100  board  scores  from  the  figure  6 
distribution.  Determine  each  score's  rank  within  the  100  board  scores. 

2.  For  each  of  these  100  scores,  randomly  assign  PTS  points  according 
to  figure  7  distribution.  Calculate  {New  Score  =  Old  Score  +  PTS)  for  each 
of  the  100  board  scores. 

3.  Determine  each  score's  new  rank  within  the  100. 

4.  For  the  selected  distribution,  perform  steps  1-3  with  20  different 
random  number  streams. 

5.  For  the  remaining  three  distributions,  repeat  steps  1-4  with  the  same 
20  random  number  streams.5 


Figure  8  illustrates  one  impact  of  awarding  PTS  points  with  these  four 
distributions.  Horizontal  axis  for  figure  8  represents  original  individual 
ranks  in  10  percent  increments  (since  the  sample  size  is  100,  each 
individual  represents  1  percent).  The  vertical  axis  is  the  average  absolute 
rank  change  (any  change  is  considered  positive)  due  to  awarding  PTS  points 
under  the  various  distributions.  For  example,  under  triangular  distribu¬ 
tion.  the  rank  of  those  in  the  original  bottom  10  changed  by  an  average  of 
just  over  two.  As  observers  would  expect,  under  the  same  distribution,  the 
average  rank  change  for  those  originally  in  the  middle  was  greater — almost 
eight.  More  difficult  to  anticipate  are  the  between-distribution  differences. 
Recall  that  all  four  options  place  a  10-percent  weight  on  PTS  (up  to  33 
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Figure  8.  Average  Impact  on  CSP  Rank 
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points).  The  point  is  that  a  10-percent  weight  can  mean  many  differenl 
things.  The  specific  distribution  selected  for  awarding  PTS  points  should 
depend  on  the  desired  impact.  All  these  example  distributions  have  at  least 
one  desirable  feature:  those  with  original  low  or  high  ranks  changed  the 
least.  Hence,  those  selected  first  under  the  current  system  would  still  likely 
be  selected.  Those  least  likely  to  be  selected  originally  are  still  least  likely 
to  be  selected.  Most  turbulence  occurs  in  the  area  where  it  is  most  difficult 
to  distinguish  between  individuals. 

On  the  surface,  there  appears  to  be  little  difference  in  impact  between  the 
“Help  High  PTS"  and  “Hurt  Low  PTS"  distributions.  Figure  9  illustrates  that 
by  another  measure,  there  is  indeed  a  difference.  The  horizontal  axis  is  the 
same  as  in  figure  8.  The  vertical  axis  lists  the  average  maximum  rank 
changes  for  the  various  distributions.  For  example,  for  the  20  triangular 
distribution  simulations,  the  average  maximum  change  in  rank  for  those 
in  the  bottom  10  is  about  seven.  The  “Help  High  PTS"  distribution  impact 
becomes  clearer.  For  those  few  with  a  high  original  rank  who  receive  33 
PTS  points,  there  is  nowhere  to  go,  and  the  maximum  rank  change  is  low. 
However,  those  few  with  a  low  or  middle  lank  who  receive  33  PTS  points 
are  able  to  move  up  in  rank  past  many  other  individuals. 

The  “Hurt  Low  PTS"  distribution  interpretation  is  similar.  For  those  few 
with  original  high  ranks  who  receive  no  PTS  points,  there  is  a  long  way  to 
fall.  For  those  originally  ranked  low  who  receive  no  PTS  points,  t  hings  can't 
get  much  worse.  Since  most  low  rankers  receive  33  PTS  points,  their 
relative  position  changes  little.  A  reason  one  might  elect  to  “Hurt  Low"  as 
opposed  to  “Help  High’  is  that  it  might  be  easier  to  identify  those  who  are 
likely  to  leave  than  those  who  are  likely  to  stay. 


ORIGINAL  RANK 

Figure  9.  Maximum  Impact  on  CSP  Rank 
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Having  said  all  this,  it  turns  out  that  for  many  production  categories, 
propensity-to-stay  points  under  any  distribution  would  not  influence  Col¬ 
lege  Scholarship  Program  decisions.  Figure  10  illustrates  the  electrical 
engineering  subset  from  figure  6.  In  this  situation.  PTS  would  not  have 
changed  scholarship  award  decisions.  All  electrical  engineer  applicants 
who  met  minimum  qualifications  were  selected.  This  pattern  is  typical  for 
electrical  engineers  because  Air  Force  needs  exceed  the  number  of  inter¬ 
ested  applicants.  A  consistent  exception  to  this  pattern  is  aerospace 
engineering,  where  applicants  exceed  needs.  For  scholarships  which  do 
not  require  a  scientific  or  technical  major,  applicants  greatly  exceed  require¬ 
ments.6 
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Figure  10.  Three-Year  Electrical  Engineering  Applicants  (February  1989  Board) 


Option  Two — Subjective 

There  are  two  ways  CSP  selection  boards  could  use  PTS  information 
subjectively.  First,  each  board  member's  maximum  score  could  remain  at 
100  points.  The  43  points  currently  based  on  AFROTC  Form  36  could 
reflect  the  PAS’s  judgment  about  PTS  (method  one).  The  alternative  would 
be  to  permit  each  board  member  to  award  up  to  11  additional  points  for 
PTS  (method  two).  This  is  explained  by  the  following  equation:  1 1  /( 100  + 
1 1)  =  10  percent.  Again,  under  method  one.  PTS  tends  to  hurt  those  with 
original  high  scores  and  help  those  with  low  scores.  Under  method  two, 
the  PTS  effect  is  independent  of  the  original  board  score.  Under  method 
one.  PTS  could  carry  a  43-percent  weight  in  extreme  cases.  Under  method 
two.  the  most  weight  PTS  could  carry  would  be  10  percent. 
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Option  Three — Combination 

Although  an  applicant's  quality  index  score  is  seen  by  PASs  on  Form  36, 
the  measure  doesn’t  carry  great  weight  in  CSP  selections.  Thus,  a  Q1S 
modification  similar  to  the  one  discussed  under  the  Weighted  ITofessional 
Officer  Course  Selection  System  would  have  very  little  impact  lor  the  College 
Scholarship  Program.  (If  the  QIS  were  modified  forWPSS  purposes,  ii  would 
necessarily  be  modified  for  CSP  purposes.)  If  up  to  1  1  objective  points  were 
allowed  for  PTS  (method  one),  and  PTS  were  also  allowed  to  influence  the 
43  subjective  points,  the  most  weight  PTS  could  carry  would  be  48.6 
percent.  This  is  explained  by  the  following  equation:  (11  +43)/(100+  11) 
=  48.6  percent.  Again,  the  impact  on  any  individual  would  depend  on 
original  rank. 

Option  Four — Optimization 

Figure  1 1  shows  an  aerospace  engineering  subpopulation  in  which 
applicants  typically  exceeded  requirements.  It  also  reflects  the  AFROTC 
policy  of  selecting  (he  highest  quality  cadels  as  measured  by  their  CSP 
score.  This  is  a  reasonable  policy.  However,  the  situation  presents  an  ideal 
opportunity  to  use  the  PTS  information.  There  are  about  20  nonselects  with 
a  CSP  score  over  191.  Ail  these  individuals  meet  minimum  requirements 
as  measured  by  electrical  engineering  standards  (see  figure  10).  If  (he  Air 
Force  had  a  retention  problem  with  aerospace  engineers,  then  the  AFROTC 
might  want  to  select  applicants  in  the  191-220  range  with  high  PTSs  over 
applicants  with  higher  CSP  scores  and  low  PTSs.  Of  course,  (he  qualified 
applicant  excess  presents  other  opportunities  to  AFROTC.  Again,  the 
obstacles  to  an  optimization  approach  are  data  development  and  building 
the  optimization  infrastructure. 
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Figure  11.  Three-Year  Aerospace  Engineering  Applicants  (February  1989  Board) 
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Four-Year  Scholarship  Program 

A  PAS  panel  of  three  reviews  each  Four-Year  Scholarship  Program 
application.  Each  officer  subjectively  assigns  each  record  a  score  of  from 
zero  to  100.  Each  member  assigns  up  to  14  points  for  academic  potential 
based  on  high  school  GPA.  class  rank,  and  course  work.  Up  to  43  points 
are  awarded  for  ext  raeurrieular  activities.  Up  to  43  points  are  also  awarded 
based  on  personal  applicant  appraisals  made  by  an  Air  Force  officer  and  by 
high  school  teachers  and  counselors.  In  addition  to  the  possible  30() 
subjective  points  awarded  by  the  board,  up  to  70  points  are  awarded 
objectively  on  the  basis  of  the  SAT/ ACT  score.7 

Boards  meet  starting  in  the  fall.  Individuals  not  selected  on  one  board 
are  not  carried  to  the  next  board.  Figure  12  graphs  score  frequencies  for 
the  FYSP  board  held  in  December  1989.  This  board  had  four  panels.  Each 
panel  reviewed  an  equal  number  of  applications  that  were  drawn  from  all 
scholarship  categories.  To  correct  for  scoring  differences  between  panels, 
the  board  interleaved  applicants  into  a  single  rank-ordered  list.  The 
number  one  ranked  individuals  from  the  four  panels  ranged  from  first  to 
fourth  on  the  composite  list.  The  individuals  ranked  second  from  the 
panels,  ranged  from  fifth  to  eighth  on  the  composite  list,  and  so  on.  Alter 
the  composite  list  was  formed,  AFROTC  managers  started  at  the  top  and 
offered  scholarships  based  on  each  applicant's  first  or  second  academic 
major  preference.  If  there  were  no  scholarships  remaining  for  either 
preference,  the  individual  was  bypassed  for  someone  further  down  the  list  .H 
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Figure  12.  December  1989  FYSP  Board  Results 
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Option  One — Objective 

The  analysis  parallels  the  CSP  option  one  analysis.  Figures  13  and  14 
summarize  results.  The  main  difference  is  that  under  method  two.  up  to 
42  points  would  be  awarded  for  PI'S.  The  equation  used  to  research  this 
conclusion  is:  42/(370  +  42)=  10  percent.  These  results  are  quite  similar 
to  CSP  findings  in  both  patterns  and  magnitudes. 

The  problem  is  that  the  original  PTS  study  used  college  experiences  to 
predict  PI'S.  Since  FYSP  applicants  are  still  in  high  school,  these  predictors 
would  not  be  available.  Therefore,  the  PTS  regression  work  would  have  to 
be  reaccomplished  before  PTS  could  be  used  in  the  FYSP.  Perhaps,  it  will 
not  be  possible  to  predict  PTS  without  college  experience  information. 

Option  Two — Subjective 

The  four-year  scholarship  program  selection  boards  could  use  PI'S 
information  subjectively  in  two  ways.  First,  each  board  members  maxi¬ 
mum  score  could  remain  at  100  points.  The  43  point''  currently  based  on 
personal  appraisal  could  reflect  the  PAS’s  judgment  about  PI'S  (method 
one).  Again,  under  method  one.  PTS  tends  to  hurt  those  with  original  high 
scores  and  help  those  with  low  scores.  In  this  case,  PI'S  could  carry  a  weight 
between  zero  and  35  percent.  This  conclusion  is  derived  by  using  the 
following  equation:  (3*431/370. 

The  alternative  would  be  to  permit  each  board  member  to  award  up  to 
14=  (42/3)  additional  points  for  PTS  (method  two).  Under  method  two, 
assuming  that  PTS  inform  ilion  would  not  influence  the  43  subjective 
points,  the  In'S  effect  would  be  independent  of  the  original  board  score. 
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Figure  13.  Average  Impact  on  FYSP  Rank 
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Figure  14.  Maximum  Impact  on  FYSP  Rank 

One  might  argue  that  this  is  too  strong  an  assumption.  Under  method  two. 
the  most  weight  PTS  could  cany  would  be  10  percent. 

Option  Three — Combination 

If  PTS  were  given  up  to  14  points  (method  one)  and  also  allowed  to 
influence  the  43  subjective  points,  the  most  weight  PTS  could  cariy  would 
be  41.5  percent.  The  fonnula  for  this  equation  is:  3*(14  +  43) /(4 1  2) . 
However,  this  would  be  an  extreme  case. 

Option  Four — Optimization 

Unlike  the  CSP  case,  FYSP  applicants  exceed  requirements  in  most 
categories.  Therefore,  PI'S  could  influence  most  selection  decisions.  Again, 
qualified  applicant  excesses  present  other  opportunities  to  AFROTC  but  for 
a  price. 


Conclusion 

AFROTC  has  at  least  four  options  and  three  decision  points  to  incorporate 
PTS  information  into  its  selection  process.  Three  options  mirror  traditional 
AFROTC  objective  or  subjective  approaches.  These  three  options  would  be 
conceptually  easy  to  implement.  The  fourth  is  an  optimization  approach 
that  would  allow  AFROTC  to  better  satisfy  operational  constraints.  Tiying 
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(o  assess  PTS  impacts  is  not  straightforward.  A  variety  of  effects  is  possible 
depending  on  implementation  methods  and  point  distribution  plans. 
Chapter  4  addresses  other  issues  that  must  be  considered  before  any  PI'S 
implementation,  including  costs  and  benefits. 


Notes 

1.  This  force  management  philosophy  is  driven  by  the  military  mission.  Fighting  wars 
requires  a  young  and  vigorous  force.  To  achieve  a  youthful,  aggressive  force,  the  military 
has  an  up  or -out  promotion  system  that  limits  the  number  of  officers  who  may  be  promoted 
and  that  also  limits  the  tenure  of  those  who  are  not  promoted. 

2.  Maj  Fred  Fisher.  Registrar.  Selections  Division  (AFROTC/RRU).  multiple  interviews 
with  author.  August  1989-April  1990. 

3.  Ibid. 

4.  Maj  Rob  Gaston.  AFROTC/RRU,  multiple  interviews  with  author.  August  1989-April 
1990. 

Using  the  same  streams  is  a  variance  reduction  technique  that  will  produce  a  more 
accurate  picture  of  the  differences  between  PTS  distribution  options. 

6.  Gaston  interview. 

7.  Maj  Mark  Lewis,  AFROTC/RRU.  multiple  interviews  with  author.  August  1989-April 
1990. 

8.  Ibid. 
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Chapter  4 


Implementation  Issues — 
Option  Four  (Optimization) 


In  addition  to  resolving  PTS  impact  questions  raised  in  chapter  3, 
AFROTC  must  consider  other  issues  before  it  implements  option  four.  The 
outlay  control  benefit  must  be  weighed  against  implementation  costs.  The 
AFROTC  scholarship  selection  system  is  geared  toward  production,  not 
outlay  control.  FYSP  allocations  are  based  on  academic  major  and  board 
rank.  Currently,  AFROTC  has  no  way  to  learn  which  school  applicants  will 
attend,  and  it  must  rely  on  historical  averages  to  estimate  what  the  FYSP 
will  cost.  For  most  production  categories,  the  CSP  selects  either  most  or  a 
few  of  the  applicants.  Consequently,  it  is  not  feasible  to  alter  outlays  by 
changing  selection  decisions.  Finally,  the  WPSS  is  a  production  control 
system  that  does  not  directly  impact  scholarship  outlays.  Unless  some 
fundamental  policy  changes  permit  AFROTC  to  control  scholarship  expen¬ 
ditures  more  directly,  a  change  to  pursue  option  four  makes  little  sense.  It 
is  not  even  clear  that  AFROTC  needs  to  manage  scholarship  outlays  on  an 
individual  basis. 


Budget  Benefits 

AFROTC  manages  its  scholarship  dollars  by  using  annual  average  costs. 
In  constant  dollars,  the  average  annual  cost  per  scholarship  for  academic 
year  1991  is  expected  to  remain  at  the  1990  level.  Alter  adjusting  for 
inflation,  AFROTC  estimated  next  year's  total  scholarship  cost  to  be  this 
year's  average  cost  mult  iplied  by  total  scholarships.  One  source  of  error  in 
this  approach  is  that  approximately  40  percent  of  next  year's  scholarship 
recipients  will  be  different  from  this  year's.  Graduation  and  attrition  cause 
this  turnover.  New  recipients  may  attend  different  schools  or  have  different 
majors.  AFROTC  can  estimate  how  much  outlay  uncertainty  results  from 
student  turnover.  Figure  15  shows  scholarship  distribution  by  cost  for 
fiscal  year  1989  with  mean  and  variance  (p.o2)  =  ($5,559,  $13,237,195). 

To  demonstrate  turnover- induced  uncertainty,  the  author  randomly  drew 
the  costs  for  5.100  scholarships  from  the  figure  15  distribution. 1  The 
author  then  calculated  total  cost  for  these  5, 100  scholarships  as  S,.  Next, 
2.040  scholarships  (40  percent  of  the  5.100)  were  randomly  replaced  to 
simulate  student  turnover.  The  total  cost  for  the  new  5.100  scholarships 
was  then  calculated  as  S2.  S,  minus  S2  is  the  constant  dollar  difference 
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INDIVIDUAL  COSTS  (.000)  (1989  DOLLARS) 

Figure  15.  Fiscal  Year  1989  Scholarship  Cost  Distribution 


between  years  due  to  turnover.  Figure  16  shows  60  between-year  S,  minus 
S2  differences.  The  sample  standard  deviation  is  $214,307.  This  means, 
in  constant  dollars,  that  about  68  percent  of  the  time  the  difference  between 
sequential-year  costs  can  be  estimated  within  ±$2 1 4.307  of  each  ot  her.  and 
that  95  percent  of  the  time,  the  difference  can  be  estimated  within 
±$428,614  (2*$214,307).  These  bounds  require  that  S!  minus  S2  be 
normally  distributed.2  In  this  case,  an  equivalent  process  to  the  one  just 
described,  let  S,  and  S2each  be  the  sum  of  .40*5. 100  =  2.040  random  draws 
from  the  figure  15  distribution.  According  to  the  central  limit  theorem,  S, 
and  S2  are  normally  distributed.  The  mean  and  the  variance  of  S,  and  S2 
are 

(2,040*$5,559,  2.040*$  1 3,237. 1 95). 

The  difference  Si  minus  S2  is  also  approximately  a  normal  distribution  with 
mean  and  variance 

($0.  4,080*$  13,237, 1 95). 

One  standard  deviation  is  $232,396.  Thus,  the  figure  16  sample  standard 
deviation  is  close  to  the  calculated  standard  deviation. 

Figure  17  shows  bounds  on  turnover- induced  variations  for  a  range  of 
scholarship  programs.  These  one  (68  percent)  and  two  (95  percent)  stan¬ 
dard  deviation  intervals  were  estimated  from  the  ligure  15  discrete  distribu¬ 
tion.  The  important  point  is  that  at  current  scholarship  levels,  the 
constant -dollar  difference  between  the  budget  and  outlays  should  rarely 
exceed  $.5  million  because  of  student  turnover. 

Another  potential  error  source  in  the  AFROTC  budget  is  the  estimation 
of  tuition  inflation  from  one  year  to  the  next.  The  fiscal  year  1990  scholar¬ 
ship  budget  was  $30,427  million.  If  the  inflation  estimate  for  the  fiscal  year 
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NUMBER 


Figure  16.  Between-Year  Total  Cost  Variations 
(in  hundreds  of  thousands  of  FY  1989  constant  dollars) 

1991  budget  were  in  error  by  1  percentage  point,  there  would  have  been  a 
$300,000  discrepancy.  Thus,  a  2-percentage  point  margin  of  error  in  the 
inflation  estimate  has  an  impact  that  exceeds  95  percent  of  the  problems 
caused  by  turnover.  In  recent  years,  tuition-inflation  rates  have  ranged 
from  5  to  7  percent.  Hence,  predicting  inflation  as  the  current  rate  would 
not  have  resulted  in  more  than  a  2-percentage  point  margin  of  error. 

A  final  factor  in  the  mismatch  between  budget  and  outlays  is  the 
numerical  difference  between  actual  and  planned  scholarship  students.  In 
fiscal  year  1989,  the  average  scholarship  cost  $5,559,  and  there  were  5,400 


Figure  17.  Turnover-Induced  Scholarship  Outlay  Uncertainty 
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planned  scholarship  students.  A  2-percent  margin  of  error  in  scholarship 
students  (108)  would  have  meant  a  $600,000  discrepancy. 

Clearly,  AFROTC  must  have  planned  student  numbers  on  scholarship. 
The  combined  effects  of  student  turnover  and  inflation  prediction  error 
should  not  have  caused  more  than  a  $1 -million  discrepancy  in  fiscal  year 
1989.  While  option  four  may  permit  better  turnover  effect  control,  AFROTC 
is  unsure  if  any  benefit  is  worth  the  cost.  To  manage  outlays,  AFROTC 
would  find  it  easier  and  more  beneficial  to  focus  analysis  efforts  on 
forecasting  inflation  and  controlling  the  number  of  cadets  on  scholarship. 

Implementation  Costs 

If  AFROTC  decides  to  implement  option  four,  it  would  have  to  develop 
and  maintain  computer  software.  This  cannot  be  done  without  incurring 
some  costs,  and  these  costs  must  be  weighed  against  benefits.  This  section 
sizes  the  problem  and  estimates  implementation  costs.  A  related  issue  is 
who  would  do  the  work.  To  understand  these  implementation  issues. 
AFROTC  must  examine  option  four’s  operational  implications. 

Ideally,  option  four  would  be  used  to  help  make  all  gray-zone  allocations 
once  each  year.  Considering  all  applicants  simultaneously  permits  maxi¬ 
mum  flexibility  in  the  WPSS.  CSP.  and  FYSP  allocation  decisions.  When 
the  applicant  pool  is  large,  it  is  easier  to  satisfy  allocation  constraints. 
However,  it  is  not  practical  to  make  all  allocations  at  one  time  because  of 
the  administrative  burden  associated  with  awarding  scholarships.  Having 
a  limited  staff  makes  it  economical  to  spread  the  administrative  effort  across 
the  year.  Another  shortcoming  of  a  single  allocation  is  that  production  goals 
change  throughout  the  year.  By  spreading  out  allocation  decisions,  current 
AFROTC  policies  allow  more  ability  to  react  to  changing  requirements.  Most 
importantly,  allocation  decisions  must  correspond  to  when  individuals 
make  personal  decisions.  Therefore,  any  contemplated  optimization  should 
initially  be  tied  to  current  AFROTC  allocation  timing.  This  would  require 
AFROTC  managers  to  establish  suboptimal  constraints. 

The  option  four  discussion  in  chapter  3  referred  to  grav-zone  production 
and  budget  constraints.  From  where  would  these  numbers  come?  AFROTC 
allocation  managers  would  have  to  generate  these  numbers  based  on  the 
current  pipeline  state  and  the  best  future  production  requirement  es¬ 
timates.  Understanding  how  managers  might  go  about  this  task  requires 
a  brief  introduction  to  steady-state  and  dynamic  models. 

Steady-State  Models 

The  scholarship  program  buys  quality.  In  theory.  AFROTC  managers 
should  meet  production  goals  at  minimum  cost  subject  to  quality  con¬ 
straints.  However,  quality  goals  do  not  exist.3  Minimum  quality  standards 
do  exist.  b\it  the  Air  Force  would  not  want  everyone  to  be  minimally 
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qualified.  How  much  is  enough?  This  question  cannot  be  answered.  In 
practice,  the  Air  Force  maximizes  quality  subject  to  budget  constraints. 

One  relevant  issue  is.  on  average,  how  many  scholarships  are  awarded 
to  meet  constant  production  targets.  For  the  purpose  of  this  discussion,  a 
steady  state  is  what  would  happen  if  everything  remained  the  same  every 
year.  At  issue  is  whether  the  AFROTC  production  pieces  theoretically  fit 
together.  Figure  18  illustrates  the  basic  idea.  Given  persistence  rates  (pi), 
various  numbers  and  types  of  scholarship  students  (Sj).  and  nonscholar¬ 
ship  students  (N|),  AFROTC  can  estimate  steady-state  production  G  and 
losses  Lj.  (Figure  18  oversimplifies,  since  it  does  not  account  for  different 
production  categories.)  For  example,  S4  entries  will  produce 
S4*pi*p2*p3*p4*p5*p6*p7*ps  graduates  in  four  years,  N2  entries  will  produce 
N2*p5*pe*p7*p8  graduates  in  two  years,  and  so  on.  Clearly,  many  Sj.  Nj 
combinations  will  produce  G  graduates  per  year.  Have  AFROTC  managers 
selected  one  of  them?  Is  it.  by  some  measure,  the  best  way?  These  are  two 
questions  which  can  be  addressed  with  a  steady-state  model.  An  additional 
steady-state  model  benefit  is  that  it  can  give  managers  insights  into  the 
direction  an  organization  might  go  to  achieve  stability  which,  in  turn, 
translates  into  efficiency  and  simplicity. 


^4  N35  N3  N2,5  n2 

Figure  18.  AFROTC  Pipeline  Steady-Slate  Model 


Dynamic  Models 

The  Air  Force  and  AFROTC  rarely  find  themselves  in  a  steady-state 
environment.  Change  is  the  rule.  AFROTC  must  have  a  dynamic  model  to 
manage  change.  The  model  would  start  with  the  current  production 
pipeline  state  and  estimate  future  states,  which  could  result  from  manage¬ 
ment  decisions.  Notionally,  such  a  model  would  look  like  the  figure  18 
model  except  that  the  p/s.  Sj’s,  Nj's,  L,'s,  and  total  production  G  change  over 
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time.  The  Sj’s  and  N/s  are  under  AFROTC  control.  However,  the  p(’s  are 
not  controllable.  Therefore,  accurate  p,  estimation  is  essential  to  forecast¬ 
ing  future  production  system  states. 

Setting  AFROTC  Production  Goals 

The  original  issue  was  how  AFROTC  managers  would  set  gray-zone 
production  and  cost  targets.  The  process  consists  of  four  steps. 

1 .  Use  a  steady -state  model  to  determine  if  average  production  goals  can 
be  achieved  within  budget  and  availability  constraints.  If  so,  set  policy  on 
the  relative  FYSP  and  CSP  production  contributions. 

2.  Use  dynamic  tools  to  help  determine  specific  courses  of  action  for  the 
current  year — how  many  scholarships  to  award  in  each  category  and  how 
much  money  these  scholarships  should  cost. 

3.  Identify,  at  each  allocation  board,  select  individuals  in  each  produc¬ 
tion  category  who  are  so  outstanding  that  PTS  does  not  influence  the 
decision. 

4.  Determine  how  many  individuals  are  required  from  the  gray  zone  to 
satisfy  remaining  production  goals. 


Size  and  Frequency  of  Problem 

Again,  an  option  four  integer  programming  (IP)  optimization  code  would 
be  exercised  each  time  an  allocation  board  convened.  This  would  occur 
about  10  times  each  year.  After  each  board  sorted  applicants  into  three 
groups,  it  would  take  about  a  week's  work  to  make  gray-zone  selections. 
AFROTC  may  have  to  investigate  “what  if  questions.  Allocating  six  weeks' 
work  per  year  would  bring  total  operational  requirements  to  four  months' 
work  per  year.4 

The  largest  applicant  pool  is  from  the  FYSP.  About  5,000  applicants  are 
considered  per  board  with  800  selections.  There  should  probably  be  three 
gray-zone  members  for  each  allocation  made.  If  200  gray-zone  allocations 
are  required,  the  total  zone  size  should  be  about  600.  It  would  be  conser¬ 
vative  to  estimate  that  1,000  decision  variables  (gray-zone  applicants) 
would  be  an  upper  bound  for  this  problem.  The  constraint  number  would 
be  less  than  50  if  AFROTC  did  not  have  widespread  viability  concerns.  This 
is  no  small  integer  programming  problem.  If  it  proves  to  be  too  difficult  to 
solve,  as  some  IP  problems  are,  heuristic  approaches  might  produce 
acceptable  solutions*  and  should  probably  be  pursued  first. 


Software  /Hardware 

One  cheap  and  quick  way  to  get  Integer  programming  software  is  to 
purchase  it  commercially.  However,  AFROTC  must  first  consider  a  number 
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of  things.  The  software  must  run  on  available  hardware,  or  other  hardware 
must  be  purchased  to  support  the  software.  Clearly,  the  hard¬ 
ware/software  decision  must  be  made  jointly.  This  particular  problem  is 
substantial  and  requires  more  than  the  average  personal  oomputcr.  barge 
integer  programming  problems  are  computationally  expensive  and  require 
fast  computers.  Air  Training  Command  leases  a  technically  attractive 
commercial  integer  programming  package  from  the  SAS  Institute.  This  SAS 
package  resides  on  an  IBM  mainframe  at  Randolph  AFB,  Texas.  AFROTC 
has  recently  purchased  a  powerful  minicomputer  that  would  be  available 
for  this  project  in  early  1991.  Again,  appropriate  IP  software  would  have  to 
be  acquired  for  this  machine. 

It  would  take  about  one  month’s  work  to  research  integer  state-of-the-art 
programming  and  to  evaluate  commercial  codes.  If  an  appropriate  com¬ 
mercial  code  can’t  be  found.  AFROTC  should  develop  a  code  in-house  that 
is  tailored  to  AFROTC’s  needs.  It  would  take  about  two  months'  work  to 
develop  an  IP  code  in-house.  Once  obtained,  an  acceptable  IP  code  requires 
little  maintenance. 


Preprocessor 

A  second  software  decision  consideration  is  that  it  must  be  possible  to 
write  a  preprocessor  for  the  IP  code.  A  preprocessor  converts  raw  data, 
such  as  that  found  in  table  3.  into  the  format  the  optimization  code  requires. 
For  small  problems,  this  task  can  be  accomplished  manually,  but  for  large 
problems,  computer  assistance  in  the  form  of  a  preprocessor  is  essential. 
Some  commercial  codes  have  intrinsic  preprocessors  that  cannot  be 
adapted  to  this  problem.  It  could  take  four  months'  work  to  develop  PI’S 
preprocessors.  Different  preprocessors  would  be  required  for  the  Four- Year 
Scholarship  Program,  the  College  Scholarship  Program,  and  the  Weighted 
Professional  Officer  Course  Selection  System  problems.  These  preproces¬ 
sors  would  require  maintenance  as  AFROTC  policy  and  environment 
change — about  every  three  months. 

Data  Maintenance 

Option  four  requires  two  basic  data  types:  information  about  individuals 
and  information  about  schools.  Required  information  about  individuals 
already  flows  to  AFROTC  data  maintainers.  AFROTC  must  extract  infor¬ 
mation  about  schools  from  information  about  groups  of  individuals. 

Most  personnel  policy  models  are  rate  driven  (p,).  These  rates  change 
over  time  and  must  be  reestimated,  at  least  annually.  A  specific  data  need 
is  for  persistence  rates  as  functions  of  academic  major,  academic  semester, 
and  academic  institution.  About  three  months’  work  is  required  to  develop 
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the  software  to  estimate  persistence  rates.  Another  two  months’  work 
would  be  required  for  annual  maintenance. 


Analysis  Support 

Writing  option  four  software  would  not  be  a  one-time  effort.  A  number 
of  things  require  changes  to  the  software  an  organization  uses.  This  is 
particularly  true  for  policy  models.  It  is  not  practical  to  build  models  that 
anticipate  all  possible  policy  questions.  Consequently,  policy  models  evolve 
over  time.  For  example.  AFROTC  might  want  to  add.  delete,  ox  change 
constraints.  At  least  three  ATC  organizations  could  develop  and  maintain 
AFROTC  software. 

AFROTC 

AFROTC  has  a  strong  interest  .a  internally  maintaining  any  software  it 
uses.  Every  commander  wants  to  explore  sensitive  issues  without  external 
scrutiny.6  Additionally,  owning  the  analysis  assets  would  permit  AFROTC 
to  set  priorities  internally.  However,  to  undertake  support  responsibility. 
AFROTC  would  have  to  develop  an  analysis  presence. 

Recruiting  Service,  Officer  Accessions 

Recruiting  service  (RS)  has  functional  responsibility  for  officer  acces¬ 
sions.  It  therefore  has  a  legitimate  interest  in  any  PTS  effort.  RS  has  a 
small  analysis  shop  at  Randolph  AFB,  Texas,  that  could  provide  support  to 
AFROTC.  However,  the  effort  could  consume  25  percent  of  the  RS  analysis 
work  force.  AFROTC  might  be  less  than  enthusiastic  about  RS  involvement 
in  internal  AFROTC  issues.  Geographic  separation  from  the  AFROTC 
headquarters  located  at  Maxwell  AFB.  Alabama,  causes  an  additional 
complication.  Close  interaction  between  analyst  and  end  user  would  be 
essential  to  develop  the  optimization  option. 

Command  Analysis  Division 

The  ATC  Command  Analysis  Division  also  has  a  small  analysis  shop.  In 
theoiy.  this  group  is  the  centralized  analysis  function  responsible  for 
providing  assistance  to  all  other  ATC  organizations.  However,  developing 
PTS  option  four  would  also  be  a  major  drain  on  resources.  Since  this  group 
has  less  direct  interest  in  officer  accessions  than  the  recruiting  service,  it 
might  be  more  acceptable  to  AFROTC.  This  group  is  also  located  at 
Randolph  AFB  and  would  have  the  same  problems  associated  with 
geographic  separation. 
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Time  Estimates 


Table  2  summarizes  time  estimates  for  option  four  development  and 
maintenance.  These  are  only  estimates.  It  probably  wouldn’t  take  more 
than  one  work  year  to  develop  option  four.  Annual  maintenance  would  vary 
between  six  and  nine  months. 


TABLE  2 


Time  Estimates 


Development 

Annual  Maintenance 

Activity 

(Months) 

(Months) 

PTS  Operations 

0 

4 

Software/Hardware 

3 

0 

Preprocessor 

4 

3 

Data 

_3 

2 

Total 

10 

9 

Conclusion 

A  decision  to  implement  option  lour  must  weigh  costs  against  benefits. 
The  prime  motivation  for  pursuing  the  optimization  option  is  improved 
control  of  scholarship  outlays.  By  managing  scholarship  turnover,  option 
four  would  bring  outlays  $500,000  closer  to  the  $30-million  scholarsl  o 
budget.  The  analysis  cost  to  the  Air  Force  would  be  about  $50,000  per  year. 
Chapter  5  draws  conclusions  on  whether  this  level  of  effort  might  yield 
greater  benefits  if  applied  differently. 


Notes 

1 .  Raw  data  provided  by  AFROTC/RRU. 

2.  Normal  distributions  are  commonly  referred  to  as  “bell-shaped  curves." 

3.  There  is  no  proven  linkage  between  traditional  measures  of  input  cjuality  and  officer 
performance.  Therefore,  Headquarters  US  Air  Force  cannot  establish  input  quality  goals 
for  AFROTC. 

4.  This  chapter's  time  estimates  are  based  on  the  author's  20  years  of  experience  in 
building  and  maintaining  simulation  models. 

5.  A  heuristic  approach  produces  near  optimal  results  with  techniques  that  are  usually 
simpler  and  faster  than  standard  approaches. 

6.  Viability  and  minority  problems  are  two  sensitive  AFROTC  issues.  Premature  con¬ 
gressional  interest  in  an  issue  can  be  vexing. 
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Chapter  5 


Summary  and  Recommendations 


Table  3  lists  obstacles  to  and  courses  of  action  to  achieve  major  AFROTC 
objectives.  This  chapter  outlines  specific  actions  AFROTC  needs  to  take  to 
better  realize  its  objectives.  As  will  be  discussed  in  greater  detail,  some 
courses  of  action  achieve  a  single  objective  while  others  achieve  multiple 
objectives. 


TABLE  3 


AFROTC  Objectives,  Obstacles,  and  Courses  ot  Action 


Objectives 

Obstacles 

Courses  of  Action 

-  To  produce 
—  Right  kind 
—  Right  number 
—  Right  time 

-  Unknown  or 

changing  persistence 
rates 

-  Control  production 
and  outlays 
—  Comprehensive 
optimization 
approach 

-  To  meet  budget 

-  Turnover-induced 
uncertainty 

-  T uition  inflation 
uncertainty 

-  Scholarship  population 
uncertainty 

-  Improve  production 
forecasting  and  adjust 
budgets  to  projected 
outlays 

—  Dynamic  model 

To  consider 
propensity  to 
stay 

-  Implementation 

-  Objective  or 
subjective  PTS 
approach 

-  To  reduce  costs 

-  Reluctance  to 
dismiss 

-  Identify  future 
losses  sooner 
—  Develop  loss  data 

Reduce  Costs 

Approximately  30  percent  of  AFROTC  tuition  dollars  is  spent  on  cadets 
who  ultimately  lose  their  scholarships.  In  1990  this  loss  will  amount  to 
about  $10  million.  One  way  to  reduce  costs  is  to  improve  applicant 
screening  so  that  fewer  scholarship  recipients  will  ultimately  lose  them. 
The  same  production  could  be  achieved  with  fewer  dollars.  This  strategy 
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requires  AFROTC  (o  improve  existing  screening  mechanisms  or  (o  develop 
new  ones.  A  second  way  to  reduce  costs  is  to  identify  and  eliminate  future 
scholarship  losers  sooner.  For  every  100  losses  eliminated  from  AFROTC 
a  year  sooner,  the  Air  Force  would  save  $500,000. 

AFROTC  needs  pertinent  scholarship  loss  data  to  implement  cost -saving 
measures.  In  the  past.  AFROTC  has  not  retained  data  on  students  who  lost 
their  scholarships.  Without  historical  data,  it  is  difficult  to  estimate 
analytically  the  probability  that  a  particular  cadet  will  graduate.  In  the 
absence  of  compelling  evidence  that  an  individual  was  a  poor  scholarship 
risk,  some  professors  of  aerospace  studies  have  been  reluctant  to  recom¬ 
mend  elimination  of  scholarship  students.1 

By  one  measure,  eliminating  marginal  students  earlier  is  preferable  to 
altering  screening  techniques.  Table  4  lists  the  relative  contribution  to  total 
scholarship  losses  by  each  class. 


TABLE  4 

AFROTC  Steady-State  Scholarship  Losses 

Freshman  Sophomore  Junior  Senior 

Percent  of  Losses  35  20  30  15 

Percent  of  Dollar  Loss  20  20  30  30 

Although  freshmen  account  for  35  percent  of  all  losses,  they  only  generate 

about  20  percent  of  the  dollar  losses.  Seniors  account  for  15  percent  of 

total  losses,  but  they  cause  about  30  percent  of  annual  dollar  losses.2  The 
explanation  for  this  paradox  is  that  freshman  losses  have  received  only  one 
or  two  semesters  of  scholarship  dollars  while  senior  losses  have  been  on 
scholarship  from  three  to  eight  semesters.  It  might  be  possible  to  predict 
which  seniors  (or  juniors)  will  ultimately  lose  their  scholarships  based  on 
freshman  and  sophomore  academic  performance.  Since  most  scholarship 
students  are  engineering  and  science  majors,  their  future  performance 
might  be  indicated  by  grades  in  freshman  calculus  and  the  so-called 
separator  class  required  early  in  most  technical  programs.3  Sufficient 
scholarship  loss  data  help  to  examine  the  strength  of  these  two  indicators. 
However,  before  any  data  collection  begins,  specific  data  requirements  must 
be  determined.  At  a  minimum,  data  should  contain  recipient's  name,  social 
security  number  (SSN).  scholarship  activation  date,  scholarship  loss  date, 
reason  for  scholarship  loss.  AFROTC  screening  scores,  name  of  university, 
and  transcript.  Preliminary  analysis  could  relv  on  a  single  year's  data.  It 
is  probably  not  necessary  for  professors  of  aerospace  studies  to  wait  for 
analysis  results.  Alter  the  sophomore  year,  the  marginal  recipient’s  profes¬ 
sors  can  probably  estimate  accurately  whether  that  individual  will 
graduate. 

AFROTC  should  begin  to  collect  scholarship  loss  data  immediately.  Since 
being  able  to  project  losses  is  not  sufficient  to  reduce  costs.  AFRO  TC  should 
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also  begin  to  install  incentives  for  professors  of  aerospace  studies  to  temper 
concent  for  their  cadets  with  cost  consideration*'  The  decision  to  remove 
students  from  scholarship  status  should  be  made  during  the  WPSS.  If  a 
scholarship  student's  only  problem  is  academic,  entrance  into  the  Profes¬ 
sional  Officer  Corps  (POC)  should  be  granted  on  a  nonscholarship  status. 
Except  for  reducing  costs,  one  can  argue  that  little  else  would  change.  It 
is  likely  the  individual  would  accept  entry  into  the  POC  in  a  nonscholarship, 
nontechnical  production  category.  Poor  grades  have  already  reduced  the 
individual’s  employment  opportunities  in  the  private  sector.  Having  lost  a 
scholarship  that  was  tied  to  an  academic  major,  the  individual  would  be 
free  to  change  to  a  more  suitable  major.  Since  the  Air  Force  Military 
Personnel  Center  (AFMPC)  does  not  usually  place  marginal  academic 
performers  into  technical  jobs,  the  ultimate  person-job  match  would  not 
change.4  Finally,  tougner  scholarship  policies  would  be  an  incentive  for 
higher  academic  performance  on  the  part  of  all  scholarship  students. 


Propensity  to  Stay 

Figure  19  summarizes  major  propensity-to-stay  (PTS)  implementation 
findings.  The  main  attraction  of  an  objective  implementation  is  predictable 
results.  A  variety  of  implementation  plans  are  possible  depending  on  the 
specific  goals  of  decision  makers.  The  most  attractive  opportunity  for  an 
objective  implementation  is  to  alter  the  quality  index  score  used  in  the 
Weighted  Professional  Officer  Course  Selection  System.  Since  the  College 
Scholarship  Program  tends  to  be  an  all-or-none  selection  process,  depend¬ 
ing  on  the  production  category.  ITS  would  have  little  real  impact  on  any 
decision.  Although  PI'S  has  the  potential  to  inlluence  Four-Year  Scholar¬ 
ship  Program  decisions,  there  is  nocurrent  PI’S  instrument  to  evaluate  high 
school  seniors,  since  the  original  PTS  regression  models  use  college  ex¬ 
periences  for  predictor  variables.  If  new  regression  equations  are  developed 
without  college  experience  information,  predictive  power  may  fall  to  unac¬ 
ceptable  levels. 

A  main  attraction  ofa  subjective  approach  to  PI'S  is  implementation  ease. 
Administratively.  AFROTC  only  needs  to  show  selection  boards  additional 
information.  The  drawback  is  that  the  weight  placed  on  PI'S  will  vary  from 
board  to  board.  The  Officer  Accession  Analysis  Division  of  Headquarters 
Recruiting  Service  (RSC)  is  working  with  AFROTC  to  assess  the  impact  of 
PI'S  in  a  subjective  implementation  environment.5  Findings  should  be 
available  in  the  summer  of  1990.  The  shortcomings  to  an  objective  PTS 
implementation  in  the  CSP  and  FYSP  also  are  present  for  the  subjective 
case. 

At  the  present  time,  any  effort  to  use  the  PI'S  to  influence  AFROTC 
decisions  should  be  focused  in  the  WPSS.  If  AFROTC  production  goals  fall, 
and  CSP  applicants  subsequently  exceed  requirements  in  most  categories, 
the  PTS  could  influence  decisions.  If  additional  research  yields  good  models 
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for  hi^fi  school  seniors,  the  ITS  could  become  pari  of  the  FYSP  seleclion 
process. 
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Meet  Budget 

Three  obstacles  cause  AFROTC  scholarship  outlays  to  deviate  from  Hie 
budget.  ICven  with  perfect  knowledge  of  tuition  inllation  and  persistence 
rates,  a  40  percent  annual  turnover  tor  those  on  scholarship  causes  an 
outlay  budget  deviation  simply  because  students  attend  different  univer 
sities.  At  a  $5,000  -st  holarship  lev<4.  the  magnitude  ol  the  deviation  should 
rarely  exceed  $500 ,000  annually.  This  deviation  is  not  completely  control 
lable  and  can  be  reduced  only  bv  <  lian^iny  AFROTC  policies  to  force 
university-student  mix  expectations. 

The  spending  devial  ion  caused  bv  a  2  pen  enlace  point  error  in  the  I  nit  ion 
inllation  forecast  exceeds  Uie  magnitude  ol  the  turnover  problem.  The 
current  AFROTC  practice  is  to  estimate  what  tuition  inllation  is  for  the 
current  year  and  then  to  use  that  rate  to  plan  for  the  future.  While 
univer  sit  ies  may  not  publish  t  nil  ion  rates  wit  h  sufficient  lead  t  imes  to  meet 
AFROTC  planning  needs,  an  estimate  of  the  coming  years  tuition  rates 
must  exist.  Professors  of  aerospace  studies  should  be  in  the  best  position 
to  obtain  this  information  for  AFK(  )T< '  plain  ’tij4  purposes. 

Kiioth  in  forecast  persistence  rates  will  also  result  in  a  difference  between 
outlays  and  budget.  For  a  $5,000  scholarship  program  and  an  average 
annual  scholarship  cost  ol  $5,000,  a  2  percentage  point  error  would  cause 
about  a  $500,000  problem  (a  deviation  ol  100  students  from  the  plan).  At 
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the  present  time,  AFROTC  persistence  rate  estimates  are  dated,  and 
calculation  methods  need  to  be  refined. 

AFROTC  needs  to  improve  its  capability  to  forecast  what  it  will  spend  and 
produce  under  various  policies.  Three  specific  requirements  stand  out. 
First,  AFROTC  needs  better  forecasts  of  tuition  inflation.  Headquarters 
AFROTC  might  encourage  professors  of  aerospace  studies  to  develop 
university  contacts  to  obtain  the  best  possible  information.  However,  the 
probability  that  professors  of  aerospace  studies  would  be  successful  ap¬ 
pears  to  be  low.  Therefore,  initial  investigations  might  be  limited  to  a  few 
universities  with  a  large  AFROTC  presence. 

Second,  AFROTC  needs  to  improve  its  persistence  rate  estimation 
methodology.  The  approach  is  conceptually  straightforward.  At  the  siaii 
of  the  academic  year,  social  security  numbers  are  retrieved  from  all  scholar¬ 
ship  students.  At  the  end  of  the  year,  social  security  numbers  of  the 
remaining  students  are  obtained  once  again.  The  ending  population, 
divided  by  the  starting  population,  is  the  persistence  rate.  Policy  decisions 
occasionally  cause  some  cadets  to  abnormally  lose  or  keep  scholarships. 
These  cadets  must  be  tracked  by  their  social  security  numbers,  which  can 
be  used  to  adjust  or  “normalize”  starting  and  ending  scholarship  popula¬ 
tions.  For  example,  an  abnormal  loss  during  the  year  would  be  added  to 
the  ending  scholarship  population  for  persistence  rate  calculations.  This 
is  a  low-risk  elfort,  and  it  should  be  pursued  without  delay. 

The  third  task  is  to  build  a  dynamic  model  that  would  serve  as  the 
analytical  tool  for  improved  outlay  and  production  forecasting.  This  is  a 
no-risk  project,  but  its  utility  will  be  limited  by  the  quality  of  inflation  and 
persistence  estimates. 


Production 

AFROTC  managers  coidd  use  the  dynamic  model  described  in  the  pre¬ 
vious  section  to  adjust  scholarship  programs  to  satisfy  externally  imposed 
production  goals  and  budget  ceilings.  The  adjustment  process  would  be 
principally  a  trial -and -error  matter,  requiring  AFROTC  to  try  different 
combinations  of  scholarship  award  programs  to  achieve  desired  results.  In 
theory,  AFROTC  should  be  able  to  use  operations  research  methods  to 
simultaneously  satisfy  production  goals,  meet  budget  constraints,  and 
select  students  with  a  high  propensity  to  stay.  In  fact,  the  original  goal  of 
this  research  was  to  build  such  a  tool.  Figure  20  illustrates  the  basic 
process.  A  selection  board  would  divide  an  applicant  pool  into  three  groups: 
the  “yes"  group  of  high-quality  applicants  to  be  selected  regardless  of  their 
FI'S;  the  “no"  group  of  low-quality  candidates  to  be  rejected  regardless  of 
their  FT'S;  and  the  “gray"  group,  whose  members  are  of  acceptable  and 
approximately  equal  quality.  After  the  selection  board  divides  the  applicant 
pool,  it  can  use  a  dynamic  model  to  forecast  the  production  and  outlay 
implications  of  selecting  the  yes  group.  The  board  can  then  rely  on  a 
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Figure  20.  Notional  Production  Control  System 


zero-to-one  integer  programming  model  to  select  from  the  gray  group 
high-PTS  students  who  would  also  meet  production  goals  and  budget 
constraints.  In  making  its  selections,  the  IP  would  consider  each 
applicant’s  PTS.  projected  tuition  cost,  and  persistence  rate  (itself  a  function 
of  semester,  major,  and  university).  The  main  attraction  of  the  optimization 
approach  is  that  AFROTC  would  increase  its  control  over  production  and 
outlays.  Unfortunately,  the  attractiveness  of  this  approach  fades  when  one 
considers  the  details  of  the  current  AFROTC  system.  Figure  21  summarizes 
the  difficulty  of  an  optimization  approach.  The  main  problem  arises  in 
trying  to  control  outlays.  Future  scholarship  expenditures  cannot  be 
controlled  through  the  WPSS  since  scholarship  decisions  are  not  made  at 
that  time.  Although  scholarship  decisions  are  made  in  the  CSP.  the 
program  tends  to  select  all  or  a  few  of  the  applicants,  depending  on  the 
production  category.  Consequently,  it  is  not  possible  to  change  outlays  by 
selecting  some  applicants  and  not  others.  The  FYSP  offers  great  oppor¬ 
tunity  to  impact  outlays.  Unfortunately,  scholarships  are  awarded  before 
an  individual's  tuition  requirements  are  known,  again  making  it  impossible 
to  control  outlays. 

In  addition  to  offering  limited  outlay  control,  optimization  requires  stable 
production  goals.  In  the  current  strength  reduction  environment.  AFROTC 
finds  it  difficult  to  predict  AFROTC  officer  production  requirements  four 
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Figure  21.  Optimization  Shortcomings 


years  hence.  Similarly,  optimization  also  requires  stable  scholarship 
budgets,  again  unlikely  for  the  next  few  years.  Finally,  optimization  re¬ 
quires  accurate  persistence  and  tuition  inflation  rates.  Clearly,  an  op¬ 
timization  approach  to  implementing  PTS,  meeting  production  goals,  and 
satisfying  budget  constraints  is  premature.  AFROTC  has  a  number  of 
higher-priority  analysis  requirements. 


Summary 

This  study  has  examined  PTS  implementation  options.  Opportunities  to 
implement  these  options  occur  while  awarding  scholarships  and  selecting 
for  the  POC.  A  variety  of  effects  can  be  engineered  with  an  objective  PTS 
implementation.  However,  effects  of  a  subjective  implementation  are  more 
difficult  to  predict.  A  PTS  optimization  approach  could  conceptually  help 
AFROTC  to  meet  budget  constraints  while  achieving  production  goals.  In 
practice,  a  combination  of  uncertainty  and  budget  constraints  reduces  the 
attractiveness  of  the  optimization  option.  There  are  other  AFROTC  analysis 
requirements  which  are  more  pressing  than  a  PTS  optimization  model.  To 
save  money,  AFROTC  needs  to  eliminate  marginal  students  earlier.  It  also 
needs  a  dynamic  model  to  improve  its  capability  to  forecast  spending  and 
production.  Nonetheless,  the  most  fundamental  need  is  for  data  to  include 
information  on  scholarship  losses  and  tuition  inflation  rates.  These  find¬ 
ings  have  been  well  received  within  AFROTC,  and  efforts  are  already  under 
way  to  implement  some  of  them. 


Notes 

1 .  During  the  course  of  (his  study,  a  number  of  Headquarters  AFRCTIV  personnel  have 
indicated  that  professors  of  aerospace  studies  tend  to  be  protective  of  their  cadets. 
Professors  of  aerospace  studies  often  argue  against  scholarship  terminations  because  the 
individuals  in  question  will  make  good  officers.  The  assumption  is  that  losing  scholarships 
will  drive  cadets  away  from  the  AFROTC  program. 
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2.  A  simple  steady-state  model  written  for  this  study  produced  these  estimates.  Chapter 
4  offered  a  discussion  of  steady-state  models. 

3.  Most  engineering  and  scientific  academic  programs  require  that  students  enroll  in  a 
“separator”  class  during  the  first  two  years  of  college.  This  class  tends  to  be  intensive  and 
intellectually  challenging.  The  department's  goal  is  to  eliminate  individuals  with  less  desire 
and  ability  early  enough  in  their  college  careers  so  they  can  switch  to  other  majors.  The 
dilemma  in  this  case  is  that,  in  changing  majors,  cadets  forfeit  their  AFROrC  scholarships. 
They  struggle  for  four  years  in  a  technical  major  only  to  be  placed  Into  nontechnical  Jobs 
because  of  poor  academic  performance. 

4.  One  of  the  Air  Force  Military  Personnel  Center's  responsibilities  is  to  assign  officers 
to  Jobs.  Low  academic  performers  are  not  generally  assigned  engineering  or  scientific  duties. 
These  cadets  are  often  placed  in  nontechnical  jobs. 

5.  In  the  spring  of  1990.  the  Officer  Accession  Division  and  AFROrC  conducted  a  test 
to  estimate  the  subjective  weight  WPSS  boards  would  give  PI'S  information.  Cadet  records 
that  had  been  scored  previously  were  presented  to  a  mock  board  with  random  PTS  scores 
assigned.  The  mock-board  results  are  currently  being  analyzed  to  see  how  much  subjective 
difference  PTS  information  made  in  individual  scores.  Headquarters  Air  Force  Recruiting 
Service,  April  1990. 
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